Cyan4973 / FiniteStateEntropy

New generation entropy codecs : Finite State Entropy and Huff0
BSD 2-Clause "Simplified" License
1.34k stars 144 forks source link

Error 39 : Decoding error : Destination buffer is too small #89

Open easterEodj opened 6 years ago

easterEodj commented 6 years ago

I tried diagnosis of the source code and using the clang-6.0 reports to find the issue for 10min, but wasn't lucky. My best guess is that the clang diagnosis is right and in lib/huf_decompress.c line 422 symbol is not assigned correctly to a U16, as it was formerly stored as a BYTE within assembling sortedSymbol in HUF_readDTableX4 line 494. Is the read out of bounds for weight and symbol on purpose?

Best Eodj

Full output: make -C programs test make[1]: Verzeichnis „/home/kvothe/Arbeit/EntropyCoder/FiniteStateEntropy/programs“ wird betreten ./probagen 20% Binary file generator Generating 1023 KB with P=20.00% File proba.bin generated compress using FSE ./fse -f proba.bin tmp FSE : Finite State Entropy, 64-bits demo by Yann Collet (Mar 29 2018) Compressed 1048575 bytes into 474414 bytes ==> 45.24%
./fse -df tmp result FSE : Finite State Entropy, 64-bits demo by Yann Collet (Mar 29 2018) Error 39 : Decoding error : Destination buffer is too small Makefile:111: die Regel für Ziel „test-fse“ scheiterte make[1]: [test-fse] Fehler 39 make[1]: Verzeichnis „/home/kvothe/Arbeit/EntropyCoder/FiniteStateEntropy/programs“ wird verlassen Makefile:36: die Regel für Ziel „test“ scheiterte make: [test] Fehler 2

Cyan4973 commented 6 years ago

Thanks for report @easterEodj .

It's a bit unexpected, as the problem seems to be triggered during the FSE decompression test, hence, no huffman function, like HUF_readDTableX4(), is supposed to be invoked.

I'm also unable currently the reproduce the issue, so I would like a few more details if possible to try to generate the error on my side.