search

snakefoot / snaketail-net

Tail utility for monitoring text log files and Windows EventLog
http://snakenest.com/snaketail/
GNU General Public License v3.0
395 stars 78 forks source link

Not usable with very large logfiles (sized several GB) #68

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
SnakeTail is (practically) unable to process very large log files. I cancelled 
trying to open a 5GB log file after 5 minutes (SnakeTail indicated "loading"; 
high CPU + Kernel times load).

Looking at the memory consumption, it does not look like the full file is 
loaded into memory, so I guess it is just read through to find the tail of the 
file (maybe also to count the number of lines in order to display a proper 
scrollbar?).

It would be much faster if the file position would be set to end of file and 
then the last n characters read. This would really enable using this tool with 
the large log files I keep encountering.

Original issue reported on code.google.com by ka...@bilby.de on 20 Dec 2013 at 12:20

brendangawn commented 7 years ago

An Enhancement, really? This app is meant (judging by the name) to give the kind of tailing performance found in typical unix systems. Given that the default behaviour of SnakeTail is to display the bottom (the "tail") for the file (exactly as I'd expect), what is the point of buffering/scanning the entire file (or whatever it is you are doing) on opening it? Might as well use a text editor. IMHO This is a bug.

snakefoot commented 7 years ago

@brendangawn You are always welcome to come with a Github PullRequest (PR) that fixes the "bug".

brendangawn commented 7 years ago

fair comment :) some day in my spare time perhaps

snakefoot commented 7 years ago

@brendangawn When I created this tool, then it was both for tailing the log file, but also to search/scroll the entire log file (not just the end of the file). Tools like Notepad, Notepad++ or LogExpert performed very poorly when the size grew beyond 50 MByte. SnakeTail can handle around 200 MByte before getting tired, which covered my needs. Applying a virtual scroll-window of the raw-text-file, that used file seek would be much faster, but then you have funny issues with current selection outside the scroll-window, and handling scrollbars to show current position as the file grows. Maybe checkout the tool Tail Blazer it seems to have cracked the nut.