Unable to generate csv programatically.

[bassamanator]

I'm unable to generate a CSV from a JSON file. My app.js is below. I'm using the code provided on the homepage. json2csv is installed as a dependency and also installed globally on my system. The CLI works fine.

1. json2csv version 5.0.5
2. Nodejs v14.15.4
3. node app.js
4. Sample and original file verified at jsonlint.com

   [{
   "ParticipantID": "a3d040db",
   "TrialType": 1,
   "TrialDuration": 1000,
   "Clicks": 3
   }, {
   "ParticipantID": "a3d040db",
   "TrialType": 2,
   "TrialDuration": 1000,
   "Clicks": 0
   }]

5. When I run the app it just outputs "ParticipantID","TrialType", nothing more. It doesn't generate CSV data the way the CLI does.

My app.js

const myFile = '/path/txxQhnxdo.json';
const fs = require('fs');

const myData = fs.readFileSync(myFile, 'utf8');
console.log(myData) // This displays file contents correctly.

const { Parser } = require('json2csv');

const fields = ['ParticipantID', 'TrialType'];
const opts = { fields };

try {
  const parser = new Parser(opts);
  const csv = parser.parse(myData);
  console.log(csv);
} catch (err) {
  console.error(err);
}

[juanjoDiaz]

Hi @bassamanator ,

You are trying to use the synchronous API which expects a JSON object but you are passing a string. To get it to work you just have to convert myData to JSON using JSON.parse(myData).

A much better approach would be to use the async API which parses the array and processes the JSON one object at a time. In the latest v5 that would be something like this (code not tested, it's just an example and it might contain some mistakes):

const myFile = '/path/txxQhnxdo.json';
const fs = require('fs');

const myData = fs.createReadStream(myFile, 'utf8');

const { AsyncParser } = require('json2csv');

const fields = ['ParticipantID', 'TrialType'];
const opts = { fields };

try {
  const parser = new Parser(opts);
  const csv = await parser.fromInput(myData).promise();
  console.log(csv);
} catch (err) {
  console.error(err);
}

[bassamanator]

Hi @juanjoDiaz ,

Parsing the string into JSON worked. Thank you so much.

I don't think I need the async API in my use case but I will read into it. The program that I'm writing is a standalone program. I just need to batch convert all my .json files into .csv files.

One serious issue that I do have at the moment is that I'm having to specify the fields. I don't want to do that because I want all the fields (there will be more than 60 of them). In the CLI I don't have to specify the fields (json2csv -i input.json -o output.csv). Can I accomplish the same in my app.js?

Thanks again!

[juanjoDiaz]

You don't have to specify them either if you use the sync API. All the fields will be used automatically.

[bassamanator]

Brilliant! Thanks again!