Closed kchodorow closed 12 years ago
-one optimization is to order the filters by how commonly they find matches, because lines are only parsed until the first time they match a filter.
Based on the logs that I had to test with, I found that the following order of occurrence is most common when it comes to messages: rs_status fsync_lock rs_sync init_and_listen stale_secondary rs_exit rs_reconfig
Perhaps I need to put stale secondary later, but that shouldn't be too much of an issue. I changed the order in which the files are parsed, and while I can't see a noticable difference in the log files I have at my disposal, the program is actually doing fewer operations.
This needs more analysis, the current speed is not good enough. Try running a code profiler on edda and figure out where it's spending most of its time.
On the 780 MB log file you gave us our time improved from about 3:40 to about 0:40, which makes close to a 500% increase in efficiency, 500% decrease in in-efficiency? Either way, I am calling this issue closed.
Nice job.
On the file Kaushal has.