oasis-open / cti-pattern-validator

OASIS TC Open Repository: Validate patterns used to express cyber observable content in STIX Indicators
https://stix2-patterns.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
26 stars 23 forks source link

Regenerate parser with ANTLR 4.9.2 #85

Closed chisholm closed 3 years ago

chisholm commented 3 years ago

Regenerated the 2.0 and 2.1 pattern parsers with ANTLR 4.9.2. The antlr4-python3-runtime python package dependency is updated to ~=4.9.0 to agree with what ANTLR itself considers a "compatible" runtime. The parsers are also now generated with -Dlanguage=Python3 since we no longer support python2, so python2-isms which were in the generated parser code are gone now, and results in some further changes (e.g. no more u"..." style unicode string literals). I also cleaned setup.py a bit: the ANTLR runtime dependency is no longer conditional based on python version; it is redundant now that only python3+ is supported. I added a python_requires to add an overall python version dependency for the library for >=3.6, which makes more sense.

codecov-commenter commented 3 years ago

Codecov Report

Merging #85 (ec9c4cb) into master (e926d0a) will decrease coverage by 0.10%. The diff coverage is 90.53%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #85      +/-   ##
==========================================
- Coverage   82.12%   82.02%   -0.11%     
==========================================
  Files          31       31              
  Lines        4874     4923      +49     
==========================================
+ Hits         4003     4038      +35     
- Misses        871      885      +14     
Impacted Files Coverage Δ
stix2patterns/v20/grammars/STIXPatternVisitor.py 0.00% <0.00%> (ø)
stix2patterns/v21/grammars/STIXPatternVisitor.py 0.00% <0.00%> (ø)
stix2patterns/v21/grammars/STIXPatternParser.py 79.62% <91.36%> (+0.15%) :arrow_up:
stix2patterns/v20/grammars/STIXPatternParser.py 80.37% <92.15%> (+0.16%) :arrow_up:
stix2patterns/v20/grammars/STIXPatternListener.py 86.15% <98.48%> (-0.36%) :arrow_down:
stix2patterns/v21/grammars/STIXPatternListener.py 97.01% <98.52%> (-0.68%) :arrow_down:
stix2patterns/v20/grammars/STIXPatternLexer.py 99.67% <99.58%> (-0.33%) :arrow_down:
stix2patterns/v21/grammars/STIXPatternLexer.py 99.67% <99.58%> (-0.33%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update e926d0a...ec9c4cb. Read the comment docs.

chisholm commented 3 years ago

The python 3.9 tests failed because it's doing pre-commit stylistic checks on generated code (the parsers). I think all style checks should be skipped for that; we should not edit it.

clenk commented 3 years ago

@chisholm can you add v21/grammars and v20/grammars to the exclusions in this file: https://github.com/oasis-open/cti-pattern-validator/blob/master/.pre-commit-config.yaml#L13? That should prevent the style checks on those files. We should add them as exclusions for the trailing-whitespace hook as well.

chisholm commented 3 years ago

Oh ok. Well, the problem wasn't that values needed to be added to the existing excludes, it was that the import sorting hook didn't have any excludes. So I added them.