aboutcode-org / scancode-analyzer

scancode-results-analyzer
4 stars 2 forks source link

Add results-analyzer plugin #32

Closed AyanSinhaMahapatra closed 3 years ago

AyanSinhaMahapatra commented 3 years ago

Adds a scancode post-scan plugin to analyze scancode license scan errors, results of which are added as scancode resource attributes. Adds tests for the plugin and analysis functions.

Fixes #20

Signed-off-by: Ayan Sinha Mahapatra ayansmahapatra@gmail.com

AyanSinhaMahapatra commented 3 years ago

@JonoYang @tdruez Thank you for reviewing! I've made the changes you requested above. @pombredanne did a lot of reviewing and restructuring over our calls, so the code format and readability are thanks to him.

AyanSinhaMahapatra commented 3 years ago

Changes made in the last force-push,

  1. Implement get_error_rule_sub_type and update ERROR_RULE_SUB_TYPE_CHOICES
  2. Better predict_license_key function [Aggregate all keys, and key with most occurrences could be the prediction, and then compare length of match if same number of occurances]
  3. Adds more tests
  4. Makes plugin more modular and better error handling
  5. Update help texts
  6. Adds commit to implement extra heuristics to address nexB/scancode-results-analyzer#29 and nexB/scancode-results-analyzer/issues/34, now with a better false-positives detection
AyanSinhaMahapatra commented 3 years ago

Changes made in the last force-push -

Other minor changes -

AyanSinhaMahapatra commented 3 years ago

@pombredanne Yes, the tests pass. I thought the travis builds were turned off so I added github action workflows. #37

Edit: No, they fail. The test expectations should change a bit because of the scancode version update (URLs change to licensedb and is_license_intro flag added). I was using the older one locally as it was failing before. I will update them.

AyanSinhaMahapatra commented 3 years ago

@pombredanne Tests pass now. The documentation test fails but that's addressed separately at #33