Options for delimiter-change (json2csv) doesn't work

[abisz]

Nothing changes when passing an options object to change the delimiter. Tried the example code and changed field and array delimiter, but the output didn't change.

Am I doing something wrong or is there a bug?

[mrodrig]

Thanks for letting me know @abisz. I've tried a few examples and it seems to be working. Could you provide me with an example of it not working so I can look into it? Thanks!

Here's what I have been trying that seems to be working:

var converter = require('json-2-csv');

var options = {
    delimiter : {
        wrap  : '\'', // Double Quote (") character
        field : ';', // Comma field delimiter
        array : ',', // Semicolon array value delimiter
        eol   : '\n' // Newline delimiter
    },
    prependHeader    : true,
    sortHeader       : false,
    trimHeaderValues : true,
    trimFieldValues  :  true,
    keys             : ['Make', 'Model', 'Year', 'Specifications.Mileage', 'Specifications.Trim']
};

var documents = [
    {
        Make: 'Nissan',
        Model: ' Murano ', // Note: This value has additional padding which can be trimmed
        Year: '2013',
        Specifications: {
            Mileage: '7106',
            Trim: ['S', 'AWD']
        }
    },
    {
        Make: 'BMW',
        Model: 'X5',
        Year: '2014',
        Specifications: {
            Mileage: '3287',
            Trim: ['XDrive', 'M']
        }
    }
];

var json2csvCallback = function (err, csv) {
    if (err) throw err;
    console.log(csv);
};

converter.json2csv(documents, json2csvCallback, options);

Output:

'Make';'Model';'Year';'Specifications.Mileage';'Specifications.Trim'
'Nissan';'Murano';'2013';'7106';'[S,AWD]'
'BMW';'X5';'2014';'3287';'[XDrive,M]'

[mrodrig]

If you're still experiencing an issue with this, please let me know. Thanks!

[abisz]

That's odd, I ran your code but I still get the wrong formatted output: Make,Model,Year,Specifications.Mileage,Specifications.Trim Nissan, Murano ,2013,7106,[S;AWD] BMW,X5,2014,3287,[XDrive;M] I use it inside a sails.js application (node version 4.4.0), but that shouldn't have anything to do with it, right?

[mrodrig]

Hmm, that's strange. I don't believe it should be a problem that it's being used inside sails.js. Do you happen to know which version you have installed? I recently changed the API to allow camel case versions of the original (all upper case) options keys, so I'm wondering if you are on an older version that didn't support them at the time.

[abisz]

I found the problem. I still had an old version (2.0.12) before you implemented the lowercase option... Sorry, my bad. It works fine now!

[mrodrig]

Oh awesome. No problem at all. Glad to hear that it is working.

[limanartem]

I just faced this issue in 5.0.1 version. Running example code outputs:

Make;Model;Year;Specifications.Mileage;Specifications.Trim Nissan;Murano;2013;7106;["S","AWD"] BMW;X5;2014;3287;["XDrive","M"]

[mrodrig]

Hi @limanartem, sorry to hear you're running into an issue. I believe the output you're getting is the expected behavior though. The default delimiter between field values (eg. "BMW" and "X5") would normally be a comma, but in the output you provided there's a semicolon (;). Since that's being specified in the options in the sample you ran, that indicates to me that the options are being respected.

Could you please provide some more details about the issue you're encountering or a data/code sample where you're finding it isn't respecting the provided options?

Also, since the module has changed a bit since the example from my 2016 comment, here's an updated code snippet for the current version (5.0.1):

const converter = require('json-2-csv');

const options = {
    delimiter : {
        wrap  : '\'', // Single Quote (') character
        field : '\t', // Tab field delimiter
        eol   : '\n' // Newline delimiter
    },
    prependHeader    : true,
    sortHeader       : false,
    trimHeaderValues : true,
    trimFieldValues  :  true,
    keys             : ['Make', 'Model', 'Year', 'Specifications.Mileage', 'Specifications.Trim']
};

const documents = [
    {
        Make: 'Nissan',
        Model: ' Murano ', // Note: This value has additional padding which can be trimmed
        Year: '2013',
        Specifications: {
            Mileage: '7106',
            Trim: ['S', 'AWD']
        }
    },
    {
        Make: 'BMW',
        Model: 'X5',
        Year: '2014',
        Specifications: {
            Mileage: '3287',
            Trim: ['XDrive', 'M']
        }
    }
];

const csv = converter.json2csv(documents, options);
console.log(csv);

which outputs:

Make    Model   Year    Specifications.Mileage  Specifications.Trim
Nissan  Murano  2013    7106    ["S","AWD"]
BMW     X5      2014    3287    ["XDrive","M"]

[limanartem]

Hi @mrodrig ,

Sorry, I didn't point out properly issue I am referring to here. I meant that "wrap" option is being ignored, not field delimiter. Even in your example output seems wrong, since "wrap" option specifies single quote delimiter, which is missing in output. Btw, also just noticed that array delimiter is ignored as well from your example

Thanks, Artem

[mrodrig]

Hi @limanartem, sorry I completely missed that initially. The reason why it isn't including the wrap delimiter is because field values are only "wrapped" with it where needed. In section 2.5 of RFC 4180 (CSV format declaration), it's noted that not all fields need to be enclosed with double quotes (default wrap delimiter), but if a quotation mark were to appear in the field value, then it must be inside double quotes. Do you have a use case where you need all of the field values to be wrapped in quotations (or a custom "wrap" delimiter option)? It should be a relatively straightforward addition if there's a need for it.

   5.  Each field may or may not be enclosed in double quotes (however
       some programs, such as Microsoft Excel, do not use double quotes
       at all).  If fields are not enclosed with double quotes, then
       double quotes may not appear inside the fields.  For example:

       "aaa","bbb","ccc" CRLF
       zzz,yyy,xxx

• https://datatracker.ietf.org/doc/html/rfc4180#section-2

Thanks for also pointing out the array delimiter bug in the example. When I was updating the code, I overlooked that option which was removed in version 3.0.0+ in favor of using JSON.stringify on arrays to help ensure that csv2json could more accurately convert the data back if needed. It's also possible to provide a function via the parseValue attribute in the options passed to json2csv to allow for a different representation of the array value to be generated. I'll edit the prior comment to remove that option from the example.

Thanks, Mike