Open Glavin001 opened 8 years ago
If you look at the scores you can see that certain classes are very difficult to predict and should be ignored.
For example, take https://github.com/Glavin001/atom-beautify
We correctly predicted 573 of 835 labelled issues, obtaining a score of 68.62275449101797%!
precision recall f1-score support add-beautifier 0.97 1.00 0.99 37 add-beautifier-option 1.00 0.93 0.97 30 add-language 1.00 1.00 1.00 144 bug 1.00 0.94 0.97 103 deprecated-atom-api 0.93 0.74 0.82 19 enhancement 1.00 1.00 1.00 134 feature 1.00 1.00 1.00 58 high priority 1.00 0.32 0.48 95 in-progress 1.00 0.36 0.53 11 invalid 1.00 0.56 0.72 66 mac 1.00 0.50 0.67 4 published 1.00 0.15 0.26 164 question 1.00 0.93 0.97 168 quick-todo 0.00 0.00 0.00 12 third-party bug 0.96 0.99 0.98 139 update third-party dependency 1.00 0.31 0.47 13 user-update-atom 0.00 0.00 0.00 4 waiting-for-user-information 0.81 0.53 0.64 32 windows 0.93 1.00 0.96 25 wontfix 0.00 0.00 0.00 7 avg / total 0.97 0.75 0.80 1265
Ignore the following labels to achieve 97%
ignore_labels = ['quick-todo', 'wontfix', 'user-update-atom', 'in-progress', 'pending-publication', 'published', 'waiting-for-user-information', 'high priority']
precision recall f1-score support add-beautifier 0.97 1.00 0.99 37 add-beautifier-option 1.00 0.93 0.97 30 add-language 1.00 1.00 1.00 144 bug 1.00 0.97 0.99 103 deprecated-atom-api 1.00 0.84 0.91 19 enhancement 1.00 1.00 1.00 134 feature 1.00 1.00 1.00 58 invalid 1.00 0.53 0.69 66 mac 1.00 0.50 0.67 4 question 1.00 0.95 0.98 168 third-party bug 0.97 0.99 0.98 139 update third-party dependency 1.00 0.31 0.47 13 windows 0.93 1.00 0.96 25 avg / total 0.99 0.94 0.96 940
Resources:
If you look at the scores you can see that certain classes are very difficult to predict and should be ignored.
Example
For example, take https://github.com/Glavin001/atom-beautify
Before
We correctly predicted 573 of 835 labelled issues, obtaining a score of 68.62275449101797%!After
Ignore the following labels to achieve 97%
Resources: