RsMetaCheck is an automated tool built to detect metadata quality pitfalls in software repositories, from version mismatches, license issues, broken URLs using SoMEF extraction capabilities. We built a catalog of the issues we documented throughout our study, classsifed in two categories: Pitfalls and Warnings based on the severity of the issue and the consensus reached between domain experts. To verify the performance of our tool we made for this study, we built a ground truth dataset based on a sample of 150 repositories sampled from a dataset of repositories that consists of InnovationSprint2025 and a small sample of openAIRE, amounting to a total of 1500 repositories, then had RSMetaCheck tested using version 0.3.0 and 0.3.5 (To evaluate the progress of our perfromance) with comparison to the ground truth that we have established.
Complete catalog of all detected metadata pitfalls and warnings with descriptions, suggestions, and examples.
→ View CatalogFull pitfall and warning detection results from manual annotation across all 150 evaluated repositories.
→ View ReportSide-by-side repository-level comparison of pitfall and warning detection between 0.3.0 and 0.3.5.
→ View ComparisonDistribution charts and grouped comparison of pitfall and warning detection across 0.3.0 and 0.3.5.
→ View Charts