mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

Extra quotes around space between separators and quotes #180

Closed quantuminformation closed 3 years ago

quantuminformation commented 3 years ago

Expected Behavior

when loading this data

csv:

"Country","Alpha-2 code","Alpha-3 code","Numeric code","Latitude (average)","Longitude (average)"
"Afghanistan", "AF", "AFG", "4", "33", "65"
"Albania", "AL", "ALB", "8", "41", "20"

I expect no extra space and quotes the in ["Alpha-3 code"] data, the same as is correctly parsed with item.Country

Actual Behavior

See:

image

How Do We Reproduce?

try parse the csv example with


 fs.createReadStream(path.resolve(__dirname, `../rawData/${fileName}`))
        .pipe(csv())
        .on("data", (data) => results.push(data))
        .on("end", async () => {
            await Promise.all(
                results.map((item) =>
                    addDbRow("countries/", item["Alpha-3 code"], { country: item.Country })
                )
            )
            console.log("All promises resolved")
        })```
quantuminformation commented 3 years ago

Looking this further appears that it doesn't parse the space correctly

"Afghanistan", "AF"

shellscape commented 3 years ago

The structure of.your CSV rows is bad. Eliminate the spaces between the quoted value and the commas and you won't have an issue. You can reference the CSV spec for more info. Other packages may handle this more elegantly, but you should always strive for a valid file.

quantuminformation commented 3 years ago

Yeh, makes sense. I uploaded to g docs and redownloaded and it is fine, thx.