search

oasis-open / cti-pattern-validator

OASIS TC Open Repository: Validate patterns used to express cyber observable content in STIX Indicators
https://stix2-patterns.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
26 stars 23 forks source link

Be flexible with antlr version? #84

Open satta opened 3 years ago

satta commented 3 years ago

I am looking to package this for Debian, and we need to depend on a given package for Python dependencies. This means that we can't always ensure a specific version, for example, for antlr4. This leads to messages like

ANTLR runtime and generated code versions disagree: 4.9.1!=4.8

all over the place with the Debian version (https://packages.debian.org/sid/python3-antlr4) vs. the one you were using. There is no way to get exactly version 4.8 there.

Is there a way to relax the requirements?

chisholm commented 3 years ago

ANTLR looks for exact match on both major and minor version values. If either is different, you get that warning message. Looks to me like their definition of "compatible version" is baked into the code there. Nothing we can really do about it. To make the generated parser compatible with runtime 4.9.x, it would have to be re-generated from the grammar files. And it would be outdated again when runtime 4.10 came out...

I wonder if it would make sense for a packager to just re-generate the parser to get something compatible with their distro's ANTLR runtime? We are slightly out of date though; @clenk would it make sense to update the master branch to 4.9.x?

satta commented 3 years ago

ANTLR looks for exact match on both major and minor version values. If either is different, you get that warning message. Looks to me like their definition of "compatible version" is baked into the code there. Nothing we can really do about it. To make the generated parser compatible with runtime 4.9.x, it would have to be re-generated from the grammar files. And it would be outdated again when runtime 4.10 came out...

OK, I see. Note that apart from the warning message, I can't see any negative effect yet. But it would be nice to sort this out to be able to package things in the correct way.

I wonder if it would make sense for a packager to just re-generate the parser to get something compatible with their distro's ANTLR runtime?

This would definitely the way to go. However, in Debian currently ANTLR and the Python runtime are already out of sync (4.7 for ANTLR vs. 4.9 for the Python runtime, see https://packages.debian.org/search?keywords=antlr4)

This would also require the package for python3-stix2-patterns to be rebuilt every time the antlr version in Debian is updated. This is doable but would require the packager to take care of this for an indefinite time, which I was hoping to avoid. But looks like there's no real way to work around it.

clenk commented 3 years ago

@chisholm yes, we should definitely update to the latest ANTLR version.

vEpiphyte commented 10 months ago

@clenk Is there anything an outside contributor can do to help updating the antlr4 version? Would a PR which has the files regenerated from an updated grammar be accepted?

rpiazza commented 10 months ago

@clenk - @vEpiphyte is volunteering to update antlr4! Can you follow thru with him?

clenk commented 10 months ago

@vEpiphyte Yes, a PR like that would be great! There's some documentation on how to do it here: https://stix2-patterns.readthedocs.io/en/latest/developers.html#updating-the-grammar and you can see the last PR that updated it here: https://github.com/oasis-open/cti-pattern-validator/pull/85

Hmm, that page on the docs uses -Dlanguage=Python2 but should be updated to use -Dlanguage=Python3 instead.

chisholm commented 10 months ago

You might also run unit tests on dependent libraries to ensure there were no changes which broke them. Two in the oasis-open organization include cti-python-stix2 and cti-pattern-matcher.

vEpiphyte commented 10 months ago

@clenk and @chisholm PR is up for this in #96