Read terminates with Out of memory when using a 2GB window size

[siara-cc]

Hi,

Thank you for making this wonderful port for ZStd!!

I am using following code:

using ZstdSharp;

byte[] ba = new byte[1000];
using var strm = System.IO.File.OpenRead(infile);
using var dstream = new DecompressionStream(strm, 131075, false);
dstream.SetParameter(ZstdSharp.Unsafe.ZSTD_dParameter.ZSTD_d_windowLogMax, 31);
Console.WriteLine("Here 1");
int iRead = dstream.Read(ba, 0, ba.Length);
Console.WriteLine("Here 2");

I am getting output:

Here 1
Out of memory.

If I don't do dstream.SetParameter(), I am getting following error:

Here 1
Unhandled exception. ZstdSharp.ZstdException: Frame requires too much memory for decoding
   at ZstdSharp.ThrowHelper.ThrowException(UIntPtr returnValue, String message)
   at ZstdSharp.DecompressionStream.Read(Span`1 buffer)
   at ZstdSharp.DecompressionStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Reddit_Read_ZStd.Main(String[] args) in /Users/arun/git/Text_frequency_research/reddit_read_zstd.cs:line 104

Any idea how this can be solved? I have tried the same using Python and C++ versions of the Zstd library and it works fine.

[oleg-st]

Hi,

I am using AllocHGlobal(int) which is limited to <2GB, need to use the nint version also.

[oleg-st]

Fixed in 0.6.3 @siara-cc Could you check it?

[siara-cc]

It is working! Thank a lot for quickly fixing this!!