search

circulosmeos / gztool

extract random-positioned data from gzip files with no penalty, including gzip tailing like with 'tail -f' !
https://circulosmeos.wordpress.com/2019/08/11/continuous-tailing-of-a-gzip-file-efficiently/
129 stars 12 forks source link

gzip tailing like tail -F #12

Closed adrfantini closed 2 years ago

adrfantini commented 2 years ago

Hi and thank you for this wonderful tool.

I was wondering if it was possible to continuously tail a gzipped file like you would with tail -F (note the capital -F, not -f!). This allows the seamless restart of the tail process if the file disappears (e.g. if it is rotated).

This would be very useful for gzipped log files that are quickly rotated. Currently, my only option, as far as I understand it, is to do something along the lines of:

while true; do
  gztool -wWTP -a1 -v0 test-log.log
done

, which is admittedly quite ugly. Do you see any other option?

circulosmeos commented 2 years ago

Hi @adrfantini

Good to hear the tool is useful! 🙂

I think the tail -F behaviour can in fact be reproduced with gztool in a future version.

The only thing to keep into account is that the behaviour will be available only with -W, because if otherwise an index file is being written, then it would have to be renamed somehow (overwriting it does not seem very sane), which seems outside gztool purposes... Ideas in this regard are welcome, nonetheless. I'll think about this in coming months.

circulosmeos commented 2 years ago

Hi, v1.5.0 just released automatically behaves like tail -F when using -[ST] with -[fW], that is, restarting the process if the gzip file is overwritten. Greetings!

adrfantini commented 2 years ago

That's awesome, thank you! Well done! This will be very useful in some cases.