search

src-d / style-analyzer

Lookout Style Analyzer: fixing code formatting and typos during code reviews
GNU Affero General Public License v3.0
32 stars 21 forks source link

Fix report metric calculation for typos analyzer #737

Closed zurk closed 5 years ago

zurk commented 5 years ago

Closes https://github.com/src-d/style-analyzer/issues/704

With these fixes I was able to achieve the next numbers:

|                    metric | value    |
|--------------------------:|:---------|
|       detection_precision | 0.702    |
|          detection_recall | 0.495    |
|   detection_true_positive | 361.000  |
|  detection_false_positive | 153.000  |
| detection_false_negatives | 368.000  |
|              fix_accuracy | 0.909    |
|         top3_fix_accuracy | 1.000    |
|                   support | 2776.000 |
|               review_time | 9.148    |

I comment on each important moment in the self-review.

Note, that I filter the dataset where typo was actually present on the expected line.

zurk commented 5 years ago

@vmarkovtsev PTAL

zurk commented 5 years ago

Updates:

  1. I rebase the PR and remove dataset update because I am going to filter it in the @EgorBu PR: https://github.com/src-d/style-analyzer/pull/741
  2. I use provided flag check_all_identifiers because there is only one commit to check, so we cannot create a set of existing identifiers.

Should be good to go.

zurk commented 5 years ago

@vmarkovtsev PTAL