Java compress data can not be decompress

[LHRchina]

I want to decompress data from java net.jpountz.lz4 compress, but failed : lz4: invalid source or destination buffer too short

[zwbdzb]

yes, I also find the question:

if bytes.EqualFold(contentEncoding, []byte("lz4")) {
        b = make([]byte, 200*1024*1024)
        _, err = lz4.UncompressBlock(remoteResp.Body(), b)
        if err != nil {
            checklist.CheckList("fetchAllDataFromMdbStore", "ReadAllDataFromMdbStoreClient-lz4-fail", checklist.Local_Ip, err.Error(), 1, 0)
            return 0
        }

debug stack:

call  len(remoteResp.Body())   18124338
call len(b)   209715200

the dst is enough larger than the encode/decode src. but still can not decode.

result in: invalid source or destination buffer too short

[tomasz1986]

I think these errors that I've encountered in Syncthing (which uses this library to compress data in transfer) could possibly be related too.

• https://forum.syncthing.net/t/connection-dropped-in-loop-with-decompressing-message-lz4-invalid-source-or-destination-buffer-too-short/18796
• https://github.com/syncthing/syncthing/issues/8484

In short, here the problem only seems to affect the i386 release, and also only occurs when decompressing specific files. The last working version is v4.1.13, while v4.1.15 is broken. There were 24 commits between https://github.com/pierrec/lz4/compare/v4.1.13...v4.1.15, so it's probably one of them that is likely the culprit.

[calmh]

Here's a test case to reproduce the failure: https://github.com/pierrec/lz4/pull/191

[tomasz1986]

I've done more testing, and v4.1.14 also seems to work fine, so the issue is down to something changed between https://github.com/pierrec/lz4/compare/v4.1.14...v4.1.15.

[tomasz1986]

It's the commit https://github.com/pierrec/lz4/commit/7e764f73f7cb74bcd418db52060b12cf76d1128e that broke it.