Empty value in CSV is converted to comma in JSON

[quillcraft]

Background Information

• Module Version: ^3.7.6
• Node/Browser Version: v10.15.3

The issue I'm reporting is with:

• [ ] json2csv
• [x] csv2json

I have...

• [x] searched to see if an issue has already been reported.
• [ ] verified that my JSON/CSV data is valid (using something like http://jsonlint.com or https://csvlint.io/).
• [x] tried upgrading to the latest version of json-2-csv (since the issue may already be fixed).

Expected Behavior

[ { param: 'c', '20/9/27': 2, '20/9/28': 2 }, { param: 'd', '20/9/27': '', '20/9/28': '' } ]

Actual Behavior

[ { param: 'c', '20/9/27': 2, '20/9/28': 2 }, { param: 'd', '20/9/27': ',', '20/9/28': '' } ]

Data Sample

CSV:

param,20/9/27,20/9/28
c,2,2
d,,

Code Example

// Please include a simple example to replicate the issue

const fs = require('fs');
const converter = require('json-2-csv');
const csvData = fs.readFileSync('sample.csv', { encoding: 'utf8' });
converter.csv2json(csvData, (error, jsonData) => {
    if (error) throw error;
    console.log(jsonData);
});

[mrodrig]

Hi @quillcraft. Thanks for reporting this. I tried the sample CSV and code provided with Node 10.15.3 and 12.18.3, but couldn't replicate the behavior you were seeing with module version 3.7.6. Could you please verify that the latest (3.7.6) version is installed? There was a fix in 3.7.5 that fixed an issue with this same scenario, so NPM might have an older version cached. If you run:

cat node_modules/json-2-csv/package.json | grep 'version'

it should show you what version is currently installed, just in case an older version is cached. I've run into that before when using ^ in front of the version number.

Here's the behavior that I saw with the example CSV/code:

[apple~ tmp ]$ node -v
v10.15.3

[apple~ tmp ]$ node test.js
[ { param: 'c', '20/9/27': 2, '20/9/28': 2 },
  { param: 'd', '20/9/27': '', '20/9/28': '' } ]

[apple~ tmp ]$ nvm use 12.18.3
Now using node v12.18.3 (npm v6.14.6)

[apple~ tmp ]$ node test.js
[
  { param: 'c', '20/9/27': 2, '20/9/28': 2 },
  { param: 'd', '20/9/27': '', '20/9/28': '' }
]

[apple~ tmp ]$ cat sample.csv
param,20/9/27,20/9/28
c,2,2
d,,

[quillcraft]

To get this behaviour run test.js several times in a row.

Archive.zip

"version": "3.7.6"

I found my mistake: there was no \n at the end in the last line of csv. I get this csv by the action of the same module (json-2-csv). And last line don't have \n at the end.

const options = {
    emptyFieldValue: '',
    delimiter: {
        wrap: '"',
        field: ',',
        eol: '\n'
    }
};
converter.json2csv(jsonDataNew, (error, csvData) => {
    if (error) throw error;
    fs.writeFileSync(file, csvData, { encoding: 'utf8' });
    console.log('Update data file: done');
}, options);

Now I have to do this:

    fs.writeFileSync(file, `${csvData}\n`, { encoding: 'utf8' });

I think this is not a very nice solution, it would be better if the converter did it.

[mrodrig]

Good find. I'll take a look to see if I can update the logic to improve the handling of that case.

[mrodrig]

I fixed the issue and published it to NPM in 3.7.7. Thanks again for reporting this.

[quillcraft]

Hi @mrodrig. Thank you very much!