Closed tatianacv closed 3 years ago
Can you provide an example?
Yes, of course. For example, from your results we get issues that are Dependabots who bump the version but do not have relevant information for our study. For example, https://github.com/RuiLiFeng/flow-gan/pull/4 which is issue 761587226, we have classifications of solution discussion, social conversation, usage, bug reproduction, potential new issues and requests, and action on issue. So we consider this issue as relevant given those classifications, but in reality is not. These classifications show up because of the release notes in the issue description.
Hmm interesting, I feel like this line should have caught it: https://github.com/ponder-lab/GitHub-Issue-Classifier/blob/main/utils/githubAPI.py#L73
Will look into this.
I think the user is 'dependabot' not 'Bot' that might be what is happening.
I'm checking for the user type, and from what the API is returning it looks like it's properly labeled as Bot
:
https://api.github.com/repos/RuiLiFeng/flow-gan/issues/4
Not sure why this would show up in the results, maybe I'll look into filtering out user.login === "dependabot[bot]"
as well. Going to look deeper into this. Will keep you posted.
Perfect, thank you!
So it turns out the original issue with https://github.com/RuiLiFeng/flow-gan/pull/4 being included in the result was because we were considering the first issue comment/description into the list of comments to be classified, but that was done separately outside of our regular check which filters out bot comments.
https://github.com/ponder-lab/GitHub-Issue-Classifier/pull/50 PR to remove comment description should have fixed this issue, but I am also further checking for dependabot's account in the login field to further make sure these types of comments don't get classified in the future.
The script is picking up Dependabot pull requests (GH API picks up open pull requests as issues) because tf.function appears in the Release Notes. Please filter out these pull requests.