Open jan-gerling opened 4 years ago
During the data validation I noticed that the detection of bug fixes based on the commit messages is sometimes incorrect. I will add some examples here:
File History: https://github.com/krishagni/openspecimen/commits/beef30d32378581db842fd19fd5437070722fb9b/WEB-INF/src/com/krishagni/catissueplus/core/common/events/AbstractListCriteria.java Relevant Commit: https://github.com/krishagni/openspecimen/commit/db4753c7470bc2190389a333e3a152354d49d2b7#diff-5ecfe971bf7c3345eb4e476482033cec Wrong Keyword: error
File History: https://github.com/xm-online/xm-ms-entity/commits/1fd7985604c9f1aeff7ff77e00860032b69ff41d/src/test/java/com/icthh/xm/ms/entity/web/rest/XmEntityResourceIntTest.java Relevant Commit: https://github.com/xm-online/xm-ms-entity/commit/a2ec1b3349b61bf5e0c841453dc6a9802b0f3711#diff-167880e1fbe9e0c65144887bf5b3e052 Wrong Keyword: Fix
File History: https://github.com/altsoft/PlatypusJS/commits/78b6e616be877a283612362f3f08b7c41471acc1/application/src/components/Forms/src/com/eas/client/forms/FormRunner.java Relevant Commit: https://github.com/altsoft/PlatypusJS/commit/0e6f5b4a769d043f0da75ed18122ff88235fbd64#diff-523abf9169c354dd326f98cf85c58e35 Wrong Keyword: Debbuger was detected as bug
We check if a commit message contains one of the keywords. This can lead to false positives: e.g. see Example 3. https://github.com/refactoring-ai/predicting-refactoring-ml/blob/fdf3753fb35b4d4e699d1f6fbb2329441d11988a/data-collection/src/main/java/refactoringml/ProcessMetricTracker.java#L22-L27
During the data validation I noticed that the detection of bug fixes based on the commit messages is sometimes incorrect. I will add some examples here:
Examples:
1.
File History: https://github.com/krishagni/openspecimen/commits/beef30d32378581db842fd19fd5437070722fb9b/WEB-INF/src/com/krishagni/catissueplus/core/common/events/AbstractListCriteria.java Relevant Commit: https://github.com/krishagni/openspecimen/commit/db4753c7470bc2190389a333e3a152354d49d2b7#diff-5ecfe971bf7c3345eb4e476482033cec Wrong Keyword: error
2.
File History: https://github.com/xm-online/xm-ms-entity/commits/1fd7985604c9f1aeff7ff77e00860032b69ff41d/src/test/java/com/icthh/xm/ms/entity/web/rest/XmEntityResourceIntTest.java Relevant Commit: https://github.com/xm-online/xm-ms-entity/commit/a2ec1b3349b61bf5e0c841453dc6a9802b0f3711#diff-167880e1fbe9e0c65144887bf5b3e052 Wrong Keyword: Fix
3.
File History: https://github.com/altsoft/PlatypusJS/commits/78b6e616be877a283612362f3f08b7c41471acc1/application/src/components/Forms/src/com/eas/client/forms/FormRunner.java Relevant Commit: https://github.com/altsoft/PlatypusJS/commit/0e6f5b4a769d043f0da75ed18122ff88235fbd64#diff-523abf9169c354dd326f98cf85c58e35 Wrong Keyword: Debbuger was detected as bug
Potential Causes:
We check if a commit message contains one of the keywords. This can lead to false positives: e.g. see Example 3. https://github.com/refactoring-ai/predicting-refactoring-ml/blob/fdf3753fb35b4d4e699d1f6fbb2329441d11988a/data-collection/src/main/java/refactoringml/ProcessMetricTracker.java#L22-L27