Reading multiple individual files from solid 7zips duplicates a lot of work

fancycode / pylzma

Python bindings for the LZMA library

http://www.joachim-bauch.de/projects/pylzma/

GNU Lesser General Public License v2.1

234 stars 72 forks source link

Reading multiple individual files from solid 7zips duplicates a lot of work #1

Closed mikelikespie closed 13 years ago

mikelikespie commented 14 years ago

Hi, it seems when you decompress stuff from solid 7zips in py7zlib it decompresses the beginning part of the file several times. Would it be possible to memoize this?

I'm going to make a patch for this.

Also, I noticed if you change self._file.read(1024) (in _read_from_decompressor) to a bigger number like 16384 there's about a 2x perf improvement.

fancycode commented 14 years ago

Thanks for the heads up, I just updated the blocksize when reading compressed files to 16384 in the repository.

fancycode commented 13 years ago

A decompression cache for solid archives has been added in 2496d6d.