Closed photopea closed 2 years ago
That file yields a 4.2 GB output. Obviously this will fail to decompress in memory. I tried to decompress with the streaming decompressor and it yielded the same output as zstd -d
(and it was actually faster, surprisingly - probably due to I/O bandwidth limitations). What code did you use to decompress?
I used fzstd.decompress(uint8array) . It seems like it is because of a bug in Google Chrome, it works fine in Firefox.
https://bugs.chromium.org/p/chromium/issues/detail?id=1201109
What about this array? Is it a valid zstd stream? I can not decode it with any decoder. It is just 1.52 MB.
It's valid and decompresses fine with the original Zstandard decompressor as well as fzstd
when done streaming. The decompressed size of that file is 2.66GB, which is still too large to decompress in memory. I think JavaScript engines in general support up to 2^31 bytes (2GiB or about 2.14GB), though Firefox should support more.
The library fails at this input. Is it a bug in a library or in the input data.txt ?