search

mattgodbolt / zindex

Create an index on a compressed text file
BSD 2-Clause "Simplified" License
620 stars 37 forks source link

Show progress when building the offsets table in verbose mode #19

Closed andreweskeclarke closed 8 years ago

andreweskeclarke commented 8 years ago

When running zindex in --verbose mode, we get the progress of the indexing step:

$ zindex some.log.gz --regex 'id:(.*)' --verbose

Building index, generating a checkpoint every 32.00 MiB
Indexing...
Progress: 20 bytes of 1.88 GiB (0.00%)
Progress: 434.48 MiB of 1.88 GiB (22.52%)
Progress: 889.66 MiB of 1.88 GiB (46.12%)
Progress: 1.32 GiB of 1.88 GiB (70.02%)
Progress: 1.73 GiB of 1.88 GiB (92.01%)
Index reading complete               <------ Can take longer than the indexing step above
Flushing
Done
Closing database

It would be nice to see a similar progress status for what is going on between "Index reading complete" and "Flushing." That step can take just as long or longer than the indexing step.