search

gendx / lzma-rs

An LZMA decoder written in pure Rust
MIT License
128 stars 27 forks source link

Streaming Decompressor #51

Closed cccs-sadugas closed 4 years ago

cccs-sadugas commented 4 years ago

Pull Request Overview

This pull request implements issue #10 .

It is currently dependant on #50 .

Testing Strategy

This pull request was tested by...

Supporting Documentation and References

This implementation is based off of the LzmaDec_TryDummy function in libhtp's port of the LZMA SDK.

TODO

Could hide this under a feature flag upon request.

Benchmarks

I added a benchmark decompress_stream_big_file that shows that streaming decompression is actually after. This could be related better memory management.

CPU: Intel(R) Core(TM) i7-1065G7 @ 1.30 GHz RAM: 16Gb @ 4267 MHz OS: Docker Windows Ubuntu 20.04 LTS

master

test compress_65536                  ... bench:   2,689,440 ns/iter (+/- 667,102)
test compress_empty                  ... bench:       1,629 ns/iter (+/- 346)
test compress_hello                  ... bench:       2,278 ns/iter (+/- 430)
test decompress_after_compress_65536 ... bench:   3,655,315 ns/iter (+/- 6,067,710)
test decompress_after_compress_empty ... bench:       3,977 ns/iter (+/- 1,797)
test decompress_after_compress_hello ... bench:       4,633 ns/iter (+/- 1,811)
test decompress_big_file             ... bench:   7,079,015 ns/iter (+/- 2,228,350)
test decompress_huge_dict            ... bench:       5,359 ns/iter (+/- 4,065)

streaming-decompressor

test compress_65536                  ... bench:   2,746,007 ns/iter (+/- 1,179,775)
test compress_empty                  ... bench:       1,618 ns/iter (+/- 1,428)
test compress_hello                  ... bench:       3,729 ns/iter (+/- 4,189)
test decompress_after_compress_65536 ... bench:   3,134,650 ns/iter (+/- 2,263,524)
test decompress_after_compress_empty ... bench:       3,747 ns/iter (+/- 1,575)
test decompress_after_compress_hello ... bench:       4,558 ns/iter (+/- 1,288)
test decompress_big_file             ... bench:   7,212,875 ns/iter (+/- 2,376,085)
test decompress_huge_dict            ... bench:       4,633 ns/iter (+/- 7,197)
test decompress_stream_big_file      ... bench:   6,670,270 ns/iter (+/- 2,605,148)
cccs-sadugas commented 4 years ago

Thanks :). I will address your comments and open a new PR.