Interoperability issue with other lz4 libraries

[raffi-semerciyan]

I'm using two github versions of lz4: yours for Golang https://github.com/jpountz/lz4-java for java.

I used your library in the code below:

package main

import (
    lz4 "github.com/bkaradzic/go-lz4"
    "fmt"
    "os"
    "encoding/base64"
)

func main() {
    var data []byte
    var err error

    to_compress:="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

    if data,err=lz4.Encode(nil,[]byte(to_compress)); err!=nil {
        fmt.Fprintf(os.Stderr,"Failed to compress: '%s'",err)
        return
    }

    fmt.Fprintf(os.Stderr,"Success! Length=%d\n",len(to_compress))

    fmt.Fprintf(os.Stdout,"%s\n",base64.StdEncoding.EncodeToString(data))
}

and obtain following output:

raffi@iot-micro-raffi ~/tmp > go run ./test_lz4.go 
Success! Length=100
ZAAAAB94AQBLUHh4eHh4
raffi@iot-micro-raffi ~/tmp > 

Then I inject this output into a Java version to decode the result:

import java.util.Base64;
import net.jpountz.lz4.LZ4Factory;
import net.jpountz.lz4.LZ4FastDecompressor;

public class test_lz4_decomp {

    public static void main(String[] args) {
        String compressed_base64= "ZAAAAB94AQBLUHh4eHh4";
        int original_size = 100;
        byte [] compressed = Base64.getDecoder().decode(compressed_base64);

        LZ4Factory _factory= LZ4Factory.fastestInstance();
        LZ4FastDecompressor decompressor = _factory.fastDecompressor();
        byte []restored = new byte[original_size];
        decompressor.decompress(compressed, 0, restored, 0,original_size);

        String decompressed_str=new String(restored);

        System.out.println(decompressed_str); 
    }

}

but obtain the following output:

raffi@iot-micro-raffi ~/tmp > java -cp ".:lz4-1.3.0.jar" test_lz4_decompException in thread "main" net.jpountz.lz4.LZ4Exception: Error decoding offset 58 of input buffer
    at net.jpountz.lz4.LZ4JNIFastDecompressor.decompress(LZ4JNIFastDecompressor.java:39)
    at test_lz4_decomp.main(test_lz4_decomp.java:16)
raffi@iot-micro-raffi ~/tmp > 

The expected result would be that the Java version should be able to decompress the output of the Golang version.

[raffi-semerciyan]

Hi, I tested with another implementation in Golang (https://github.com/pwaller/go-clz4) and it worked.

So I believe there is a problem in the way the compression is performed in your library.

Thanks anyway for your efforts and contribution!

[bkaradzic]

LZ4 algorithm used here is from 2011. There were some significant changes in LZ4 since then (for example streaming support). I don't expect that it works with newer LZ4 versions.