search

KillingSpark / zstd-rs

zstd-decoder in pure rust
MIT License
253 stars 34 forks source link

don't return errors on too large requests on a reversed bitreader #58

Closed KillingSpark closed 5 months ago

KillingSpark commented 7 months ago

Removing the checks for too large requests (> 56 bits) on a reversed bitreader allows for a whole bunch of error handling in hot loops to be removed which leads to better performance.

Doing this allows more weird values to float around requiring more careful handling of the returned values in many places, more fuzzing is definitely necessary.

KillingSpark commented 6 months ago

This should include a change that checks the decoded FSE tables so that the symbols do match the allowed ranges for literal length, match length and offset which would avoid weird values appearing and would also detect corrupted files earlier and more consistently.

(And the code for creating these tables is currently very messy, a cleanup in this module would be nice anyways)