mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

Headers not applied #143

Closed GianlucaRi closed 4 years ago

GianlucaRi commented 4 years ago

Hello ! I have no problem with this library in all the cases except one : When I had custom header. Maybe I am just doing a mistake

Expected Behavior

Headers present in json.

Actual Behavior

It still return 0, 1, 2 ... has headers

How Do We Reproduce?

test.csv

AFG356;20190927;116;108;187;275;;;127;14;113;148;47;101;100;100;;;;;;;;
AFG372;20190927;54;52;79;;;;;;;;;;;;;;;;;;;
AFG380;20190927;609;385;751;206;;7;34;34;;165;94;71;665;665;;;;;;;;
AFG414;20190927;103;53;67;73;;7;33;33;;33;26;7;89;89;;;;;;;;
AFG439;20190927;98;55;98;97;;1;53;15;38;43;34;9;65;65;;;;;;;;
AFG515;20190927;610;557;812;253;7;12;87;;87;8;7;1;485;485;;;;;;;;
AFG515;20190928;610;599;971;194;14;13;34;;34;11;;11;416;416;;;;;;;;
AFG557;20190927;126;112;187;167;;9;68;;68;80;21;59;175;175;;;;;;;;
AFG557;20190928;126;82;158;131;2;13;;;;91;18;73;185;185;;;;;;;;
AFG562;20190927;73;36;56;84;;;;;;84;41;43;42;;;42;;;;;;
AFG562;20190928;73;40;62;81;;;;;;81;36;45;49;;;49;;;;;;

index.js

const folder = "/path/to/folder";

const csv = require('csv-parser');
const fs = require('fs');

const results = [];
fs.createReadStream(folder + '/test.csv')
    .pipe(csv({
            headers: ["AAA", "BBB", "CCC", "DDD", "EEE", "FFF", "GGG", "HHH", "III", "JJJ", "KKK", "LLL",
                "MMM", "NNN", "OOO", "PPP", "QQQ", "RRR", "SSS", "TTT", "UUU", "VVV", "WWW"],
            separator: ";",
        }))
    .on('data', (data) => results.push(data))
    .on('end', () => {
        console.log("*********** results size" + results.length);
        console.log(results[0]);
        console.log(results[1]);
    });
shellscape commented 4 years ago

@GianlucaRi thanks for the issue. Please put your reproduction into a repo that can be cloned and run without any additional manual steps being required by maintainers.

I've added an explicit test to make sure the option is working correctly:

It would appear that it's something on your end at this time.

GianlucaRi commented 4 years ago

Thank you, with your test I have found the problem.

It was a length header problem, I have to put headers with a good size.

If I do this, it works :

const filepath = "./test.csv";

const csv = require('csv-parser');
const fs = require('fs');

const results = [];
fs.createReadStream(filepath)
    .pipe(csv({
        // work 
        headers: ["AAA", "BBB", "CCC", "DDD", "EEE", "FFF", "GGG", "HHH", "III", "JJJ", "KKK", "LLL", "MMM", "NNN", "OOO", "PPP", "QQQ", "RRR", "SSS", "TTT", "UUU", "VVV", "WWW", "XXX"],
        // work with more
        // headers: ["AAA", "BBB", "CCC", "DDD", "EEE", "FFF", "GGG", "HHH", "III", "JJJ", "KKK", "LLL", "MMM", "NNN", "OOO", "PPP", "QQQ", "RRR", "SSS", "TTT", "UUU", "VVV", "WWW", "XXX", "YYY"],
        // don t work with less header
        // headers: ["AAA", "BBB", "CCC", "DDD", "EEE", "FFF", "GGG", "HHH", "III", "JJJ", "KKK", "LLL", "MMM", "NNN", "OOO", "PPP", "QQQ", "RRR", "SSS", "TTT", "UUU", "VVV", "WWW"],
        separator: ";",
    }))
    .on('data', (data) => results.push(data))
    .on('end', () => {
        console.log("*********** results size" + results.length);
        console.log(results[0]);
        console.log(results[1]);
    });

I'd rather not create a repository just for that, but I have modify the javascript file to allow make the local testing easy. The data must be in the same folder name : "test.csv", and the code above in index.js npm install csv-parser node index.js

Thank you for your help,

Have a nice day !

shellscape commented 4 years ago

glad you got it sorted!