Very big difference between "lz4c -BD" and "lz4c"

[GoogleCodeExporter]

What steps will reproduce the problem?

I tried to check compression speed on my linux by compressing my system 
partition.

# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        19G   12G  5.9G  67% /

What is the expected output? 

I expected same result or little difference

What do you see instead?

# lz4c64 -v -BD /dev/sda1 > /dev/null 
*** LZ4 Compression CLI 64-bits v1.0.6, by Yann Collet (Oct 22 2013) ***
Blocks size : 4096 KB
Using stdout for output
Compressed 19999490048 bytes into 18403981488 bytes ==> 92.02%                 
Done in 27.92 s ==> 683.13 MB/s

# lz4c64 -v /dev/sda1 > /dev/null  
*** LZ4 Compression CLI 64-bits v1.0.6, by Yann Collet (Oct 22 2013) ***
Blocks size : 4096 KB
Using stdout for output
Compressed 19999490048 bytes into 5959944348 bytes ==> 29.80%                  
Done in 43.98 s ==> 433.67 MB/s

What version of the product are you using?

# lz4c -V
*** LZ4 Compression CLI 64-bits v1.0.6, by Yann Collet (Oct 22 2013) ***

On what operating system?

uname -a
Linux gentoo-rlaptop 3.7.10-gentoo #12 SMP Thu Apr 18 23:51:41 MSK 2013 x86_64 
Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz GenuineIntel GNU/Linux

Original issue reported on code.google.com by roman.st...@nexenta.com on 23 Oct 2013 at 12:14

[GoogleCodeExporter]

Thanks for notifying.
This behavior is not expected.
I'll investigate.

Original comment by yann.col...@gmail.com on 23 Oct 2013 at 1:19

[GoogleCodeExporter]

I tested the behavior, and can confirm it.

It seems that, in -BD mode, nothing got compressed after crossing the 4GB 
limit. Looks like an overflow error...

Original comment by yann.col...@gmail.com on 23 Oct 2013 at 1:36

• Changed state: Accepted
• Added labels: Priority-Medium
• Removed labels: Priority-minor

[GoogleCodeExporter]

The issue can only happen in 64-bits mode, when initial buffer address is 
allocated above 4GB, and the amount of data streamed is > 4GB.

The attached file is supposed to solve this issue.

Original comment by yann.col...@gmail.com on 23 Oct 2013 at 3:15

Attachments:

• lz4.c

[GoogleCodeExporter]

works.

also i can confirm that this bug is related to 64bit version.

Thanks.

Original comment by roman.st...@nexenta.com on 23 Oct 2013 at 3:39

[GoogleCodeExporter]

Corrected into r108

Original comment by yann.col...@gmail.com on 2 Nov 2013 at 12:12

• Changed state: Fixed