While trying to decompress the same file using ZstdDirectBufferDecompressingStream
I'm getting the same error and I'm failing to find a way to pass a --long or a --memory option
java.io.IOException: Frame requires too much memory for decoding
at com.github.luben.zstd.ZstdDirectBufferDecompressingStreamNoFinalizer.read(ZstdDirectBufferDecompressingStreamNoFinalizer.java:89)
at com.github.luben.zstd.ZstdDirectBufferDecompressingStream.read(ZstdDirectBufferDecompressingStream.java:66)
File f = new File("hello.txt.zst");
FileChannel fileChannel = FileChannel.open(f.toPath(), StandardOpenOption.READ);
MappedByteBuffer fileBuffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
int size = 1024 * 50;
ByteBuffer streamBuffer = ByteBuffer.allocateDirect(size);
while (streamBuffer.hasRemaining() && fileBuffer.hasRemaining()) {
streamBuffer.put(fileBuffer.get());
}
streamBuffer.flip();
ZstdDirectBufferDecompressingStream zis = new ZstdDirectBufferDecompressingStream(streamBuffer) {
@Override
protected ByteBuffer refill(ByteBuffer toRefill) {
if (fileBuffer.hasRemaining()) {
toRefill.clear();
while (toRefill.hasRemaining() && fileBuffer.hasRemaining()) {
toRefill.put(fileBuffer.get());
}
toRefill.flip();
}
return toRefill;
}
};
ByteBuffer block = ByteBuffer.allocateDirect(1024 * 100);
while (zis.hasRemaining()) {
block.clear();
zis.read(block);
block.flip();
byte[] out = new byte[block.remaining()];
block.get(out);
System.out.println(new String(out, StandardCharsets.UTF_8));
}
I have tried to use a Zstd.setCompressionLong (and getting stream pointer from com.github.luben.zstd.ZstdDirectBufferDecompressingStreamNoFinalizer#stream using reflection) but i was keep getting the same error.
try to decompress the file without --long=31
try to decompress with --long=31 option. Success!
While trying to decompress the same file using ZstdDirectBufferDecompressingStream I'm getting the same error and I'm failing to find a way to pass a --long or a --memory option
I have tried to use a Zstd.setCompressionLong (and getting stream pointer from
com.github.luben.zstd.ZstdDirectBufferDecompressingStreamNoFinalizer#stream
using reflection) but i was keep getting the same error.