compress data size larger than 64KB,LZ4_decompress_safe() failure

[GoogleCodeExporter]

What steps will reproduce the problem?
1.MEMORY_USAGE is 14.
2.compress a 524288 byte data block,and this data is compressed to 526108.
3.Call LZ4_decompress_safe, decompress  fail(retrun -63133).

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
the tested version is r110
os: suse 11 sp1, x86_64

Please provide any additional information below.

if the size of compress data block is less than 64KB,or MEMORY_USAGE is 15, or 
run on a 32-bit system, LZ4_decompress_safe()can decompress correct.

here is my test program:

#include <stdlib.h>
#include <stdio.h>

#define BLOCK_SIZE_MAX (1024 * 1024)
static char in[BLOCK_SIZE_MAX];
static char out[LZ4_COMPRESSBOUND(BLOCK_SIZE_MAX)];
static char uncomp[BLOCK_SIZE_MAX];

int main(void)
{
    struct stat buf;
    FILE *inf;
    int rsize, fsize;
    int csize, dsize;
    void *lz4state;

    stat("dumpFile", &buf);
    fsize = (int)buf.st_size;

    inf = fopen("dumpFile", "r"); // dumpFile is a 512KB file
    if(NULL == inf)
    {
        fprintf(stderr, "open fail\n");
        return -1;
    }

    lz4state = malloc(LZ4_sizeofState());
    if(NULL == lz4state)
    {
        fclose(inf);
        return -1;
    }

    while((rsize = fread(in, 1, BLOCK_SIZE_MAX, inf)) > 0)
    {    
        csize = LZ4_compress_withState(lz4state, in, out, rsize);
        if(csize > 0)
        {
            //dsize = LZ4_decompress_fast();
            dsize = LZ4_decompress_safe(out, uncomp, csize, rsize);
            if(dsize < 0)
            {
                fprintf(stderr, "decompress fail, rsize=%d, csize=%d, dsize=%d\n",
                        rsize, csize, dsize);
                break;
            }
        }
        else
        {
            fprintf(stderr, "compress fail, rsize=%d, csize=%d\n",rsize, csize);
            break;
        }
    }

    fclose(inf);
    free(lz4state);

    return 0;
}

Test Results：decompress fail, rsize=524288, csize=526108, dsize=-63133

sorry, my english is poor .

Original issue reported on code.google.com by nicksun1...@gmail.com on 10 Jan 2014 at 2:50

[GoogleCodeExporter]

Does the problem also happen when using LZ4_compress(), or is it specific to 
LZ4_compress_withState() ?

Original comment by yann.col...@gmail.com on 10 Jan 2014 at 4:16

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

so sorry.
if MEMORY_USAGE is 14, LZ4_compress() and LZ4_compress_withState() has the 
poblem.
but LZ4_decompress_fast(out, uncomp, rsize) decompress correctly.

Original comment by nicksun1...@gmail.com on 10 Jan 2014 at 6:40

[GoogleCodeExporter]

OK, thanks for report.
So it happens with MEMORY_USAGE == 14, system is 64-bits, with data compressed 
by LZ4_compress() or LZ4_compress_withState(). The problem is only 
visible/triggered with LZ4_decompress_safe().

Is it possible to have access to a sample (data to compress) that produces the 
problem ? This would help much for debugging.

Regards

Original comment by yann.col...@gmail.com on 10 Jan 2014 at 7:24

[GoogleCodeExporter]

I've been trying to reproduce the issue, without success so far.
"dumpFile" seems to be incompressible, or hardly compressible.
Such case is part of the standard non regression test suite.
I've checked it again, and tried to find manually some corner cases, without 
success so far.

So I guess, if it was possible to have access to "dumpFile", it would help to 
correctly observe the problem.

Original comment by yann.col...@gmail.com on 10 Jan 2014 at 11:21

[GoogleCodeExporter]

I am sorry that the problem was caused by another process(Kill the process,the 
test case run correctly).
So sorry.

Original comment by nicksun1...@gmail.com on 15 Jan 2014 at 11:42

[GoogleCodeExporter]

I'm glad you could find the cause of your bug.

Original comment by yann.col...@gmail.com on 16 Jan 2014 at 12:39

[GoogleCodeExporter]

Original comment by yann.col...@gmail.com on 16 Jan 2014 at 12:39

• Changed state: Invalid