adaltas / node-csv-parse

CSV parsing implementing the Node.js `stream.Transform` API
https://csv.js.org/parse/
804 stars 166 forks source link

Error parsing quoted cell with new lines and comma. #140

Closed jwerre closed 6 years ago

jwerre commented 7 years ago

I download some a csv from Google Contacts and linted it to make sure it was valid but I'm getting an error when I parse it in versions 1.2.0 and 1.1.12. If I open it in a spreadsheet app and export it again the problem is resolved.

I believe there error has something to do with the new lines in the address cell:

"62 Corporate Park Ste 225
    Irvine, CA 92606
    Orange County"

outlook_contacts.csv

First Name,Middle Name,Last Name,Title,Suffix,Initials,Web Page,Gender,Birthday,Anniversary,Location,Language,Internet Free Busy,Notes,E-mail Address,E-mail 2 Address,E-mail 3 Address,Primary Phone,Home Phone,Home Phone 2,Mobile Phone,Pager,Home Fax,Home Address,Home Street,Home Street 2,Home Street 3,Home Address PO Box,Home City,Home State,Home Postal Code,Home Country,Spouse,Children,Manager's Name,Assistant's Name,Referred By,Company Main Phone,Business Phone,Business Phone 2,Business Fax,Assistant's Phone,Company,Job Title,Department,Office Location,Organizational ID Number,Profession,Account,Business Address,Business Street,Business Street 2,Business Street 3,Business Address PO Box,Business City,Business State,Business Postal Code,Business Country,Other Phone,Other Fax,Other Address,Other Street,Other Street 2,Other Street 3,Other Address PO Box,Other City,Other State,Other Postal Code,Other Country,Callback,Car Phone,ISDN,Radio Phone,TTY/TDD Phone,Telex,User 1,User 2,User 3,User 4,Keywords,Mileage,Hobby,Billing Information,Directory Server,Sensitivity,Priority,Private,Categories
Parkview,,Dental,,,,,,,,,,,,parkview@gmail.com,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,(949) 555-0001 ,,"62 Corporate Park Ste 225
Irvine, CA 92606
Orange County",62 Corporate Park Ste 225,,,,Irvine,CA Orange County,92606,,,,,,,,,,,,,,,,,,Normal,,My Contacts,
Paul,,Bunyan,,,,,,12/24/1915,,,,,,paul-bunyan@yahoo.com,paul-bunyan@gmail.com,paul-bunyan@outlook.com,,,,555-4715,,,"123 Ox Dr., Edmonds WA 96734","123 Ox Dr., Edmonds WA 96734",,,,,,,,,,,,,,+1284 234-12331,,,,,,,,,,,"47658 Pike Street., Seattle WA 96734","47658 Pike Street.,
96734",,,,Seattle,WA,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,My Contacts;Heros,
Pecos,,Bill,,,,,,5/12/1989,,,,,,pecos@yahoo.com,p@gmail.com,,,,,+48 555-5445,,,"1234 Trim Rd., Faketown PA 99999","1234 Trim Rd., Faketown PA 99999",,,,,,,,,,,,,,555-8778,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,My Contacts;Heros,
Peter,Pumkin,Eater,,,,,,3/29/1977,,,,,,peter@pumkin-eater.com,peter_pumkin@hotmail.com,,,555-8897,,555-1234,,,"23 W 736 Oak St., Edmonton Ca 99949","23 W 736 Oak St., Edmonton Ca 99949",,,,,,,,,,,,,,555-7485,,,,,,,,,,,"234 Willow Rd., Roselle Il, 60172",234 Willow Rd.,,,,Roselle,Il,60172,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Normal,,My Contacts;Heros,

test.js

let csvParse = require('csv-parse');

fs.readFile(__dirname + '/contacts_outlook.csv', 'utf8', function(err, data) {
    csvParse(data, function(err, result) {
        console.log(err, result);
    });
});

error:

Error: Number of columns is inconsistent on line 4
    at Error (native)
    at Parser.__push (/xxx/node_modules/csv-parse/lib/index.js:233:13)
    at Parser.__write (/xxx/node_modules/csv-parse/lib/index.js:453:14)
    at Parser._transform (/xxx/node_modules/csv-parse/lib/index.js:181:10)
    at Parser.Transform._read (_stream_transform.js:167:10)
    at Parser.Transform._write (_stream_transform.js:155:12)
    at doWrite (_stream_writable.js:334:12)
    at writeOrBuffer (_stream_writable.js:320:5)
    at Parser.Writable.write (_stream_writable.js:247:11)
    at /xxx/node_modules/csv-parse/lib/index.js:46:14
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)
wdavidw commented 7 years ago

Please simplify your test to the maximum before submitting an issue, it takes hours for us to track and identify the problem otherwise.

alexdrans commented 6 years ago

Any update on this @jwerre? I had the same issue, then I updated from 1.2.1 to 1.2.3 and the issue went away. I assume this commit fixed it. Close @wdavidw ?

wdavidw commented 6 years ago

The commit you are mentionning only apply when the relax option is set to "true". I dont think it was the case in this issue. However, let's close the issue and anyone is free to re-open it.