circulosmeos / gztool

extract random-positioned data from gzip files with no penalty, including gzip tailing like with 'tail -f' !
https://circulosmeos.wordpress.com/2019/08/11/continuous-tailing-of-a-gzip-file-efficiently/
133 stars 12 forks source link

Action create index for a gzip from STDIN not working #14

Closed aitorarjona closed 2 years ago

aitorarjona commented 2 years ago

Hello, thanks for maintaining this amazing tool :)

I need to use the reading from stdin feature, but is not working anymore for version 1.5.0. I tried for version 1.4.3 and is working fine.

Steps to reproduce:

$ git clone https://github.com/circulosmeos/gztool.git
$ git fetch
$ git checkout v1.5.0
$ automake --add-missing && autoreconf && ./configure && make check
$ cat tests/gplv3.txt.gz | ./gztool -I index.bin
ACTION: Create index for a gzip file

Processing STDIN ...
Processing index to 'index.bin'...
ERROR: Compressed data error in STDIN.
$ make clean
$ git checkout v1.4.3
$ automake --add-missing && autoreconf && ./configure && make check
$ cat tests/gplv3.txt.gz | ./gztool -I index.bin
ACTION: Create index for a gzip file

Index file 'index.bin' already exists and will be used.
Processing STDIN ...
Processing index to 'index.bin'...
$ ./gztool -ell index.bin
ACTION: Check & list info in index file

Checking index file 'index.bin' ...
    Size of index file (v1)  : 84.00 Bytes (84 Bytes)
    Number of index points   : 1
    Size of uncompressed file: 31.27 kiB (32024 Bytes)
    Number of lines          : 207
    Compression factor       : Not available
    List of points:
    #: @ compressed/uncompressed byte L#line_number (window data size in Bytes @window's beginning at index file), ...
#1: @ 20 / 0 L1 ( 0 @60 ), 

1 files processed

Thank you!

circulosmeos commented 2 years ago

Hi @aitorarjona ,

Great thanks for your feedback and your kind words ! 😊

You're right, v1.5.0 fails in that case: I think the process is correctly done, but gztool fails to display the correct "ok" message when reaching the end of the file.

I've just released v1.5.1 which reverts v1.5.0's STDIN changes, so now everything should run as smoothly as before. v1.5.0 STDIN changes are little noticeable (only when tailing small inputs), and if I find spare time and a way to patch this incorrect behaviour I'll probably release a new version with that code again.

Any other feedback with v1.5.1 is welcome! 👍

Greetings!