search

huikau / logstalgia

Automatically exported from code.google.com/p/logstalgia
0 stars 0 forks source link

High number of log entries per second. freezes Logstalgia. #68

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
If using a log file with more than 700 log entries per second will cause 
Logstalgia to freeze.

What is the expected output?
Hopefully that Logstalgia was able to handle this and a lot more.

What do you see instead?
Once Logstalgia reaches the part of the log that contains a massive amount of 
traffic it freezes.

What version of the product are you using?
Logstalgia v1.0.3

On what operating system?
Xubuntu 14.04

Please provide any additional information below.
I noticed this when trying to visualize a iptables log file that had captured 
an outbound DDoS attack. The average number of entires per second was 93708. I 
started removing lines per second until i found a limit where Logstalgia didnt 
freeze up. I still had to go down to around 500-600 lines per second to get is 
viewable.
I fully accept that this might be "a bit" above the limits of Logstalgia but, 
is there any way to tweak Logstalgia to accept a bit more than this?

Original issue reported on code.google.com by are@bifrozt.com on 26 Nov 2014 at 2:54

GoogleCodeExporter commented 9 years ago 
First thing would be to try the current version since 1.0.3 is really old.

1.0.5 is definitely a lot faster but not many orders of magnitude faster.

Cheers

Andrew

Original comment by acaudw...@gmail.com on 26 Nov 2014 at 8:54

GoogleCodeExporter commented 9 years ago 
Yep, i know 1.0.3 i ancient :P

Pulled the latest one (1.0.7) from your github now and testing its limits.

//Are

Original comment by are@bifrozt.com on 26 Nov 2014 at 5:30

GoogleCodeExporter commented 9 years ago 
Working a lot better in 1.0.7, still have to shave off a lot of lines during 
extremely high traffic peaks but, its working a lot better indeed.

Thanks :)

//A

Original comment by are@bifrozt.com on 26 Nov 2014 at 6:03

GoogleCodeExporter commented 9 years ago 
Tried again with 1.0.7 on a machine with a lot more resources.
Logstalgia is now running smoothly if the log entries/second is kept under 4500.
You can deff close this ticket.

Original comment by are@bifrozt.com on 27 Nov 2014 at 9:28

GoogleCodeExporter commented 9 years ago 
Ok. Good to know it handles a lot better! :)

Original comment by acaudw...@gmail.com on 27 Nov 2014 at 9:42

GoogleCodeExporter commented 9 years ago 
Yep, sure does :)
....Now if we only could get it to accept a bit more, maybe +50.000 
entries/second, that would be great :D

Original comment by are@bifrozt.com on 27 Nov 2014 at 10:15