Premature end of data while decompressing

[abhisheksurve45]

Hi team,

I'm getting Premature end of data while decompressing the byte array

java.io.IOException: Premature end of data
     at org.apache.commons.compress.utils.ByteUtils.fromLittleEndian(ByteUtils.java:138) ~[commons-compress-1.21.jar!/:1.21]
     at org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorInputStream.nextBlock(FramedLZ4CompressorInputStream.java:249) ~[commons-compress-1.21.jar!/:1.21]
     at org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorInputStream.init(FramedLZ4CompressorInputStream.java:172) ~[commons-compress-1.21.jar!/:1.21]
     at org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorInputStream.<init>(FramedLZ4CompressorInputStream.java:112) ~[commons-compress-1.21.jar!/:1.21]
     at org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorInputStream.<init>(FramedLZ4CompressorInputStream.java:96) ~[commons-compress-1.21.jar!/:1.21]
     at net.test.demo.serializer.redis.RedisCacheLZ4Serializer.decompress(RedisCacheLZ4Serializer.java:32) ~[common-1.0.0.jar!/:?]
     at net.test.demo.serializer.redis.RedisTemplateSerializer.deserialize(RedisTemplateSerializer.java:31) [common-1.0.0.jar!/:?]
     at org.springframework.data.redis.core.AbstractOperations.deserializeHashValue(AbstractOperations.java:355) [spring-data-redis-2.5.10.jar!/:2.5.10]
     at org.springframework.data.redis.core.DefaultHashOperations.get(DefaultHashOperations.java:55) [spring-data-redis-2.5.10.jar!/:2.5.10]

Decompress function:

public byte[] decompress(byte[] input) throws IOException {
        try (ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(input)) {
            try (FramedLZ4CompressorInputStream framedLZ4CompressorInputStream =
                    new FramedLZ4CompressorInputStream(byteArrayInputStream)) {
                try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
                    framedLZ4CompressorInputStream.transferTo(byteArrayOutputStream);
                    return byteArrayInputStream.readAllBytes();
                }
            }
        }
    }

Compress function:

public byte[] compress(byte[] input) throws IOException {
        try (ByteArrayOutputStream byteArrayOutputStream =
                new ByteArrayOutputStream(input.length)) {
            try (FramedLZ4CompressorOutputStream framedLZ4CompressorOutputStream =
                    new FramedLZ4CompressorOutputStream(byteArrayOutputStream)) {
                framedLZ4CompressorOutputStream.write(input);
                framedLZ4CompressorOutputStream.flush();
                return byteArrayOutputStream.toByteArray();
            }
        }
    }

This is being used with JdkSerializationRedisSerializer

@Override
    public Object deserialize(byte[] bytes) {
        try {
            return super.deserialize(decompress(bytes));
        } catch (IOException e) {
            throw new SerializationException(e.getMessage());
        }
    }

Serialization working fine with snappy / zstd

Snappy:

public byte[] decompress(byte[] input) throws IOException {
        try (ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(input)) {
            try (SnappyFramedInputStream snappyFramedInputStream =
                    new SnappyFramedInputStream(byteArrayInputStream)) {
                try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
                    snappyFramedInputStream.transferTo(byteArrayOutputStream);
                    return byteArrayInputStream.readAllBytes();
                }
            }
        }
    }

Apache common utils version: https://mvnrepository.com/artifact/org.apache.commons/commons-compress/1.21

Same issue is happening with LZ4FrameInputStream from https://mvnrepository.com/artifact/org.lz4/lz4-java/1.8.0

Please let me know if anything else required to debug the same. Thanks again.

[abhisheksurve45]

@odaira

[abhisheksurve45]

getting

 java.io.IOException: Stream ended prematurely
     at net.jpountz.lz4.LZ4FrameInputStream.readInt(LZ4FrameInputStream.java:245) ~[lz4-java-1.8.0.jar!/:?]
     at net.jpountz.lz4.LZ4FrameInputStream.readBlock(LZ4FrameInputStream.java:259) ~[lz4-java-1.8.0.jar!/:?]
     at net.jpountz.lz4.LZ4FrameInputStream.read(LZ4FrameInputStream.java:353) ~[lz4-java-1.8.0.jar!/:?]
     at java.io.InputStream.transferTo(Unknown Source) ~[?:?]

with LZ4FrameInputStream from https://mvnrepository.com/artifact/org.lz4/lz4-java/1.8.0

[abhisheksurve45]

This is solved

public byte[] decompress(byte[] input) throws IOException {
        try (ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
                InputStream inputStream = new ByteArrayInputStream(input)) {
            try (InputStream snappyInputStream = new LZ4FrameInputStream(inputStream, false)) {
                IOUtils.copy(snappyInputStream, outputStream);
            }
            return outputStream.toByteArray();
        }
    }