Crash with too big data

[Frankusky]

1. Include the version of json2csv used. 4.5.2

2. Include your node version/browser vendor and version. v10.13.0

3. Include the command or code you used. json2csv.parse(dbQueryResult, {delimiter: '\t', header: false, fields: tableColumns});

4. Include a sample dataset that we can test against. My dataset has 22 objects, and for this case it crashed when the query returned 881950 elements

[ {data1: 1, data2: 2, data3: 3, data4: 4, data5: 5, data6: 6..... (till 22)}, {data1: 1, data2: 2, data3: 3, data4: 4, data5: 5, data6: 6..... (till 22)} ...(till 881950) ]

5. Include your output/error. <--- Last few GCs --->

[8416:000001AA50C92E70] 441325 ms: Mark-sweep 1393.4 (1425.5) -> 1393.1 (1426.0) MB, 1497.7 / 0.0 ms (average mu = 0.093, current mu = 0.002) allocation failure scavenge might not succeed [8416:000001AA50C92E70] 442625 ms: Mark-sweep 1393.8 (1426.0) -> 1393.5 (1426.5) MB, 1297.1 / 0.0 ms (average mu = 0.048, current mu = 0.002) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 0000019AD07DC5C1]
1: StubFrame [pc: 0000019AD07DDB86]

Security context: 0x03d44ba931e1 2: processCell [000003C0E785B589] [{my directory}\node_modules\json2csv\dist\json2csv.cjs.js:~1298] [pc=0000019AD0B3A3F8](this=0x030015b2e0f9 ,row=0x02fd1fdb65f9

[knownasilya]

The issue you're experiencing is due to pulling all of the data into memory. Most database libraries/orms have a streaming feature which should be used instead. Have a look at the streaming API docs here: https://github.com/zemirco/json2csv#json2csv-async-parser-streaming-api

[juanjoDiaz]

What he said ^

And feel free to ask here with more details of what you are trying to do if you need help.

[Frankusky]

Thanks for the quick support guys! It worked perfectly :) Closing