search

horsicq / DIE-engine

DIE engine
MIT License
2.3k stars 320 forks source link

Crypto pattern searching looks very slow #40

Open greenozon opened 3 years ago

greenozon commented 3 years ago

on big files (50-100 MB) it took very long time to scan for crypto sigs...

image

questions: does the tool use multi-threads to speed up the search? what kind of algo does it use to search sigs does it use any of the industrial mentioned here https://en.wikipedia.org/wiki/String-searching_algorithm eg: Boyer–Moore

I understand the term "slow" is very relative, but even on my modern machine (multicore, multi threaded) it takes 5..10 minutes to complete this action)

horsicq commented 3 years ago

It uses for scan 8 threads :)

greenozon commented 3 years ago

Great news How about algo? :)

ideally the SW should use O(n) complexity

https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm yet another power algo!

horsicq commented 3 years ago

Thanks a lot for the links. I will take a look.