search

oasis-open / cti-pattern-matcher

OASIS TC Open Repository: Match STIX content against STIX patterns
https://github.com/oasis-open/cti-pattern-matcher
BSD 3-Clause "New" or "Revised" License
44 stars 20 forks source link

Add filter generator #31

Open jmgnc opened 7 years ago

jmgnc commented 7 years ago

Figure out how, given a pattern to generate a corse filter for the pattern. Like only observables w/ a cyber object that has type file.

This would let us pre filter the observable data before being passed to the matcher to help make it faster.

gtback commented 7 years ago

We've (and by "we" I really mean @chisholm) started working on something like this in the "pattern-validator" repo, which is quickly turning into a more general purpose library for STIX 2 patterns.

https://github.com/oasis-open/cti-pattern-validator/blob/master/stix2patterns/inspector.py

We haven't yet integrated the pattern-validator into this repo, though, so we'll probably want to do that as well.

gtback commented 7 years ago

@jmgnc : Now that stix2-patterns is a dependency of this library, can you talk more about what types of filters you'd want to support? Or maybe this issue is better handled in the other repo.

jmgnc commented 7 years ago

We can handle it here, or maybe move the issue to the other repo.

The main ones I can think of are the requirement for specific types. This won't help us out too much as most of our observed-data will be network-traffic, then the next one would be what extensions are required.

I did realize that this does cause problems for != checks, as that can match any non-specific types. Hope this is a good start.

gtback commented 6 years ago

@jmgnc, I'm trying to figure out what the next step is for this issue. The matcher does now depend on the stix2patterns library, so we can make use of the inspector. But I don't know what a "filter" should look like. We also need to define how patterns contain "!=", or an "OR" combining known and unknown object types, should behave. We probably also need to figure out what the correct "status" is for observations that cannot be evaluated are.

I'd like to specify some of these things before we get too far into an implementation. That said, if this is no longer a need, we can probably just close this.