LZ4_uncompress_unknownOutputSize can return incorrect decompressed size

[GoogleCodeExporter]

What steps will reproduce the problem?
1. call LZ4_uncompress_unknownOutputSize with an osize that is exactly the size 
of the uncompressed data. It will return a decompressed size that is too small.

What is the expected output? I expected 65535 What do you see instead? I got 
65531

The bug is in line 534. The line following a memcpy.  ip += length; Since 
LZ4_uncompress_unknownOutputSize uses op to calculate the length unless there 
is an error this line should be op += length;

Original issue reported on code.google.com by Clayton....@gmail.com on 26 Oct 2011 at 7:05

[GoogleCodeExporter]

Actually I expected 65536 because that is the block size a was using. I had a 
typo above.

Original comment by Clayton....@gmail.com on 26 Oct 2011 at 7:06

[GoogleCodeExporter]

Hi Clayton
Thanks for reporting the issue.

Just to clarify : in your sentence "with an osize that is exactly the size of 
the uncompressed data", in which field are you providing osize ?
Is that maxOutputSize=osize ?

Regards

Original comment by yann.col...@gmail.com on 26 Oct 2011 at 7:52

[GoogleCodeExporter]

I'll handle this one.

Original comment by yann.col...@gmail.com on 26 Oct 2011 at 7:52

• Changed state: Accepted

[GoogleCodeExporter]

Yes. I tried to write from memory. Instead of osize I meant maxOutputSize.

Original comment by Clayton....@gmail.com on 26 Oct 2011 at 7:58

[GoogleCodeExporter]

This behavior is confirmed. Both the problem and the solution seems to be 
exactly as you report. The bug seems corrected in latest r37.
Clayton, could you please test it and tell if it works for you too ?
Regards

Original comment by yann.col...@gmail.com on 26 Oct 2011 at 8:17

[GoogleCodeExporter]

Thanks. This compressor is fast. I have only been using it as a C# port. I 
changed it in that and it is working well.

Original comment by Clayton....@gmail.com on 26 Oct 2011 at 10:48

[GoogleCodeExporter]

C# port ? Is it an open source project ?

Original comment by yann.col...@gmail.com on 26 Oct 2011 at 11:12

[GoogleCodeExporter]

Original comment by yann.col...@gmail.com on 26 Oct 2011 at 11:12

• Changed state: Fixed

[GoogleCodeExporter]

Well, the source code isn't out anywhere. I can send you a copy of what I have 
if you want it. It was mostly a matter of replacing some of the #defines with 
the const keyword and then I manually expanded all the macro functions since I 
didn't trust C# to inline them and in the past copy and paste inlining has made 
a considerable speed difference. Maybe that could be done with T4 templates but 
I've never used them. Memcpy was replaced with some C# code. Then I added a 
couple extra functions so I could pass byte arrays in and get them back but 
still use pointers inside. I only modified the LZ4.c file not the demo file. I 
could make it a little better and release it on google code or something but 
I'm not sure I would maintain it. If I have time I'll look at using T4 
templates or something else for the macro expansion so it will look more like 
the c file and be easier to update if the original file changes. By the way I 
also ported LZ4HC to C# but that was harder because some of the code couldn't 
be translated directly so it looks a little less like the original. It is quite 
a bit slower than LZ4 also.

Original comment by Clayton....@gmail.com on 27 Oct 2011 at 7:46

[GoogleCodeExporter]

Sounds like quite a great job you achieved. And LZ4HC is quite complex, so i 
guess it was a thorough translation.
Well, if anytime you feel like it would be a good idea to open source it, i'll 
be glad to link it from LZ4 Homepage if you wish.

Original comment by yann.col...@gmail.com on 27 Oct 2011 at 8:02