search

ulikunitz / xz

Pure golang package for reading and writing xz-compressed files
Other
484 stars 45 forks source link

feat: added limitedByteReader to avoid breader #56

Open orisano opened 1 year ago

orisano commented 1 year ago

The ByteReader function in some cases creates a breader that reads one byte at a time, resulting in inefficient performance. However, if the passed Reader implements io.ByteReader, it is more efficient to use it instead.

In the case of LZMA2, it is wrapped with io.LimitReader, which inadvertently triggers the usage of breader. To avoid this, I have added an io.ByteReader with the functionality of io.LimitedReader. This change has shown approximately a 10% performance improvement.

I kindly request your review and feedback on this pull request. Thank you.

ulikunitz commented 1 year ago

Many thanks for your PR. I have to review it in detail and will do it this week.

orisano commented 1 year ago

benchstat:

goos: linux
goarch: amd64
pkg: github.com/bodgit/sevenzip
cpu: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
             │  base.txt   │             xz#55.txt              │    xz#55_xz#56_sevenzip#95.txt     │
             │   sec/op    │   sec/op    vs base                │   sec/op    vs base                │
LargeLZMA2-8   10.775 ± 0%   8.704 ± 0%  -19.22% (p=0.000 n=10)   8.384 ± 0%  -22.19% (p=0.000 n=10)
ulikunitz commented 1 year ago

Hi, would it be possible for you to adapt the change for the rewrite branch? The new branch supports multithreading and has a number of other performance improvements.