search

gendx / lzma-rs

An LZMA decoder written in pure Rust
MIT License
129 stars 27 forks source link

Limit memory use for huge dictionary #7

Closed gendx closed 4 years ago

gendx commented 7 years ago

The current implementation of LZBuffer allocates the full dictionary size in the constructor, even if the decompressed file is much smaller, which can slow down decompression for small files. Use a dynamic strategy to grow the dictionary on-demand instead, or check the decompressed size in the header (if provided).

gendx commented 7 years ago

Added some tests in 43d806c4474491d30cd36709afb2961362978046, it seems that vec![0; size] already grows on-demand when data is accessed (at least on Linux). But this may not be the case on other platforms, and the allocation could fail if there is not enough RAM, so a fix is still welcome.

gendx commented 4 years ago

Fixed by #22.