search

Legilibre / legi.py

Outils de manipulation des archives LEGI (lois françaises)
56 stars 19 forks source link

Show progress bar while loading tarballs #83

Closed mdamien closed 3 years ago

mdamien commented 4 years ago

Since the loading takes a lot of time on my computer (it should take ~2 days), I added a progress bar :smiley:

Changaco commented 4 years ago

I don't think reading the archive twice is an acceptable solution, it's too inefficient.

mdamien commented 4 years ago

Gonna try to fix that, doing a simple ˋtqdm(list(archive))` didn’t work (another error appeared) so I made this hack

Changaco commented 4 years ago

A tarball doesn't have an index, so there's no way to know how many files it contains without reading it entirely. However, if we could get the byte offset of the current entry within the archive, then we could have a proper progress bar.