mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

Support of line comma suffix was removed. #155

Closed shlomo666 closed 4 years ago

shlomo666 commented 4 years ago

Expected Behavior

Used to read files just fine when files had "," at the end of every line, back in my old version 1.12.0.

Actual Behavior

Creating a map of values where the keys are 0-N (numbers of columns).

How Do We Reproduce?

File:

a1,b1,c1,
a2,b2,c2,

code:

const csv = require('csv-parser');
const fs = require('fs');
const results = [];

fs.createReadStream('b')
  .pipe(csv({ headers:["a", "b", "c"],  } ))
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
    console.log(results.length);
  });

Result in v2.3.1:

[ { '0': 'a1', '1': 'b1', '2': 'c1', '3': '' },
  { '0': 'a2', '1': 'b2', '2': 'c2', '3': '' } ]

Result in v1.12.0:

[ Row { a: 'a1', b: 'b1', c: 'c1' },
  Row { a: 'a2', b: 'b2', c: 'c2' } ]
shellscape commented 4 years ago

The CSV Specification does not specifically address trailing commas and doesn't specify that they should be dropped. Empty/missing values between or after commas are treated as an empty column of data.

shlomo666 commented 4 years ago

Correct. Thanks.

mapo-job commented 4 years ago

Having the same issue. Solve it by doing : delete record['']; just don't do if(record['']) but if(record[''] != undefined)

Also I get different result when running on different os, same node version. For "csv-parser@2.3.0"

Example:

//data.csv content
//NAME,AGE,
//Daffy Duck,24,
//Bugs Bunny,22,

const csv = require('csv-parser')
const fs = require('fs')
const results = [];

fs.createReadStream('data.csv')
.pipe(csv({mapValues: ({ header, index, value }) => isNaN(+value)?value:+value}))
.on('data', (data) => results.push(data))
.on('end', () => {
  console.log(JSON.stringify(results[0], null, 2));
});

Linux (Red Hat 4.8.5-36) node version v12.16.1 result: { "NAME": "Daffy Duck", "AGE": 24, "": 0}

Windows 10 node version v12.16.1 result: { "NAME": "Daffy Duck", "AGE": 24}