cloudflare / wirefilter

An execution engine for Wireshark-like filters
https://blog.cloudflare.com/building-fast-interpreters-in-rust/
MIT License
958 stars 85 forks source link

Any method to combine multi filter into one to accelerate? #83

Open hxndg opened 8 months ago

hxndg commented 8 months ago

Hello, I'm now writing a package filter demo using wirefilter master branch, with almost 10rules using regex.

I'm curious is there any method to accelerate the match process by combinng multi filter into one to accelerate the process, rather than compare each rule(filter) sequency?

veeshi commented 8 months ago

If you are just looking to match if any of the rules matched then you can combine them with ands but I assume you want information on which of them matched, this isn't possible as a filter when executed only returns a boolean.

hxndg commented 8 months ago

If you are just looking to match if any of the rules matched then you can combine them with ands but I assume you want information on which of them matched, this isn't possible as a filter when executed only returns a boolean.

Combine with " or " , I guess your meaning? For now , just check if any match without match rule id is acceptable.

But if I combine with or, does it do accelerate the check speed? Filter execution latency can be a problem

veeshi commented 8 months ago

Yep, my bad, combine with or not and! I think it may be slightly faster, the execution context is passed by reference and a filter is just a boxed Fn. If you don't require the rule ID which matched I'd recommend combining them but benchmark first to compare the execution times.

We execute hundreds of rules with minimal latency issues on our platform, looping through rules and executing them but YMMV. We have found using the builtin operators sometimes being faster than regex matches but that depends on your use case and the matching being done.