Cannot decode this file

[photopea]

I can not decode these binary bytes: https://photopea.com/zstdBytes.bin . Could you fix it somehow?

[101arrowz]

That file decompresses to a file 3.7GB large, and it cannot be decompressed in memory with JavaScript for the same reason as this: https://github.com/101arrowz/fzstd/issues/9. You can, however, decompress this file by streaming the data; here's an example in Node.js:

import { Decompress } from 'fzstd';
import { createReadStream, createWriteStream } from 'fs';

const src = createReadStream('zstdBytes.bin');
const dst = createWriteStream('test');
const dec = new Decompress();

let lastBW = 0;

dec.ondata = (chk, final) => {
    dst.write(chk);
    if (dst.bytesWritten > lastBW + 100_000_000) {
        console.log(Math.floor((lastBW += 100_000_000) / 100_000_000) + '00MB')
    }
    if (final) {
        dst.end();
        console.log(`Done, wrote ${dst.bytesWritten} bytes`)
    }
}

src.on('data', chk => dec.push(chk));
src.on('end', () => dec.push(new Uint8Array(0), true));

This fully decompresses zstdBytes.bin to test, even though zstdBytes.bin decompresses to a file >3GB. This is because it processes the data in 64kB chunks, one at a time. The key part is Decompress and dec.push(chunk, final) - you can adapt this to a web application quite easily, let me know if you'd like an example.

[101arrowz]

Let me know if this is still an issue when you stream the decompressor; closing for now.