search

adaltas / node-csv

Full featured CSV parser with simple api and tested against large datasets.
https://csv.js.org
MIT License
3.98k stars 263 forks source link

csv-parse fails to parse very large CSV files #416

Closed mmarkell closed 7 months ago

mmarkell commented 7 months ago

Describe the bug

I get this error when I attempt to parse a 2GB file:

Cannot create a string longer than 0x1fffffe8 characters

I assumed there was chunking happening under the hood to break this up, but based on the error I assume there is some code that pulls the whole file into memory.

To Reproduce

pass a very large file to csv-parse