mrodrig / json-2-csv

Convert JSON to CSV *or* CSV to JSON!
https://mrodrig.github.io/json-2-csv
MIT License
421 stars 58 forks source link

[Feature] Streaming chunk option for large files #263

Open WillDelish opened 2 months ago

WillDelish commented 2 months ago

I'm curious if there are any recommended ways to use this library on large data sets?

When building out a .csv, I've hit over 2.5gb of RAM usage spike with some giant JSON files.

Being able to stream writes in chunks to a .csv would be a way to avoid such a high spikes, if this could be a feature request.

Thanks for this amazing library! So easy to use.