Closed OTLabs closed 4 years ago
I run lzbench (master branch) on NanoPi NEO2 aarch64 board to test all zstd compression levels on Silesia compression corpus.
Everything runs fine till I get to level 22, at this point lzbench gets killed without any error message.
Running zstd on same file with compression level 22 produces error message.
Would quite appreciate if you can incorporate error message in lzbench for low memory condition.
I would quite appreciate your comments on memory requirements estimation for each compression levels also.
Here I would like to share some output logs:
otlabs@nanopineo2:~$ free total used free shared buff/cache available Mem: 1011064 80988 723160 11536 206916 846836 Swap: 505528 13564 491964 otlabs@nanopineo2:~/dev/lzbench$ zstd --ultra -22 silesia.tar zstd: error 11 : Allocation error : not enough memory otlabs@nanopineo2:~/dev/lzbench$ zstd --ultra -21 silesia.tar silesia.tar : 24.88% (211957760 => 52743949 bytes, silesia.tar.zst) otlabs@nanopineo2:~/dev/lzbench$ ./lzbench -ezstd,22 silesia.tar lzbench 1.8 (64-bit Linux) Assembled by P.Skibinski Compressor name Compress. Decompress. Compr. size Ratio Filename memcpy 675 MB/s 769 MB/s 211947520 100.00 silesia.tar Killed otlabs@nanopineo2:~/dev/lzbench$ ./lzbench -ezstd,22 lzbench18_sorted.md lzbench 1.8 (64-bit Linux) Assembled by P.Skibinski Compressor name Compress. Decompress. Compr. size Ratio Filename memcpy 3429 MB/s 4788 MB/s 14940 100.00 lzbench18_sorted.md zstd 1.4.4 -22 0.59 MB/s 104 MB/s 3573 23.92 lzbench18_sorted.md done... (cIters=1 dIters=1 cTime=1.0 dTime=2.0 chunkSize=1706MB cSpeed=0MB)
Try the following options: -b# set block/chunk size to # KB (default = MIN(filesize,1747626 KB)) -m# set memory limit to # MB (default = no limit)
I run lzbench (master branch) on NanoPi NEO2 aarch64 board to test all zstd compression levels on Silesia compression corpus.
Everything runs fine till I get to level 22, at this point lzbench gets killed without any error message.
Running zstd on same file with compression level 22 produces error message.
Would quite appreciate if you can incorporate error message in lzbench for low memory condition.
I would quite appreciate your comments on memory requirements estimation for each compression levels also.
Here I would like to share some output logs: