oleg-st / ZstdSharp

Port of zstd compression library to c#
MIT License
200 stars 29 forks source link

Frame requires too much memory for decoding (+ fix) #37

Closed KoalaBear84 closed 1 month ago

KoalaBear84 commented 1 month ago

I've had some issues with decompression a single file, other went fine.

My fix was the following:

using FileStream fileStreamInput = File.OpenRead("TheInputPath");
using FileStream fileStreamOutput = File.OpenWrite("TheOutputPath");

using var decompressionStream = new DecompressionStream(fileStreamInput);
// This is the fix
decompressionStream.SetParameter(ZstdSharp.Unsafe.ZSTD_dParameter.ZSTD_d_windowLogMax, 31);
decompressionStream.CopyTo(fileStreamOutput);

Took the 31 from this ZSTD_WINDOWLOG_MAX_64: https://github.com/facebook/zstd/blob/v1.4.5/lib/zstd.h#L1049

I have no idea what it all means, but I hope it helps someone stuck with the same issue 😃

KoalaBear84 commented 1 month ago

Cross posted solution to other lib: https://github.com/skbkontur/ZstdNet/issues/32

oleg-st commented 1 month ago

This seems to work as it should. See https://facebook.github.io/zstd/zstd_manual.html#Chapter6 for a description of the ZSTD_d_windowLogMax parameter.

KoalaBear84 commented 1 month ago

I don't say that anything is wrong 😅 Only issue it wasn't working and I had no idea, found a solution and wanted to let it know for others 😇

Ahh, powers of 2, that makes sense why it's only 31 and not the real amount bytes. 👍

oleg-st commented 1 month ago

Okay, thank you.