decoder-leco / poc-eurostat-data-transformers

A first data transformers POC for Eurostat data
https://decoder-leco.github.io/poc-eurostat-data-transformers/index.html
0 stars 0 forks source link

May we have a pattern here (streaming ?) #9

Open Jean-Baptiste-Lasselle opened 5 months ago

Jean-Baptiste-Lasselle commented 5 months ago

It's not just about extremely large files, but about streaming data (which does not necessarily involves http downloding a file): https://dev.to/wesleymreng7/processing-massive-amount-of-data-on-demand-without-crashing-nodejs-main-thread-2epl

https://github.com/wesleymreng7/file-stream-loader/tree/main/src