mayhemer / logan

Log Analyzer for Mozilla logs
7 stars 4 forks source link

Optimize processing #8

Closed mayhemer closed 7 years ago

mayhemer commented 7 years ago

split and map rules by modules and by [word, separator, word, the rest] so that we don't have to iterate the whole rule array over every line in the log - O(n x m).

mayhemer commented 7 years ago

https://github.com/mayhemer/logan/commit/1a986a4e1d31edd8d329fa567ff8eaa7080fbdc2

But despite we are iterating only 0.5% of rules as before the effect is small - saving some 20-25% of time only.