ChunkingRangeError: Array buffer allocation failed

[danny900714]

I upload my file with w3 CLI. If I upload large file, it will show such error.

Output:

# Packed 1237 files (42683.2MB)
# bafybeihrgb6hche5r6khltriaecihyhwl5spoe7yl2243qwvehaq62nuby
⠙ ChunkingRangeError: Array buffer allocation failed
    at new ArrayBuffer (<anonymous>)
    at new Uint8Array (<anonymous>)
    at read (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/node_modules/@ipld/car/esm/lib/decoder.js:166:20)
    at async Object.exactly (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/node_modules/@ipld/car/esm/lib/decoder.js:183:9)
    at async readBlock (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/node_modules/@ipld/car/esm/lib/decoder.js:86:17)
    at async Object.blocks (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/node_modules/@ipld/car/esm/lib/decoder.js:113:15)
    at async decodeReaderComplete (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/node_modules/@ipld/car/esm/lib/reader-browser.js:53:20)
    at async Function.put (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/node_modules/web3.storage/src/lib.js:116:19)
    at async put (file:///C:/Users/User/AppData/Roaming/npm/node_modules/@web3-storage/w3/index.js:142:16)

[olizilla]

How much RAM is available on the system you're using?

That output shows it managed to chunk and hash your file to get the root CID, but it then failed to put the parts, probably when it when to try and split the CAR into ~10MiB chunks.

We have a fix for this that will make the entire upload pipeline streaming, I will link to an issue for that when we have one.

[danny900714]

There is 12GB RAM for my PC.

I don't encounter this issue when using Go client. Maybe the implementation of counting root CID cause this error.