adaltas / node-csv

Full featured CSV parser with simple api and tested against large datasets.
https://csv.js.org
MIT License
4.05k stars 267 forks source link

Parser errors on escaped quotes that are not quoted #435

Closed evanrh closed 2 months ago

evanrh commented 2 months ago

Version

csv-parse 5.5.5

Error

CsvError: Invalid Opening Quote: a quote is found on field 0 at line 1, value is "The "

Description

The parser errors out on columns where there is an escaped quote, but the column itself is not quoted.

Expected Behavior

The record would be parsed as normal with the quotes being escaped in the output string.

Steps to Reproduce

import { parse } from "csv-parse/sync";

const line = `The ""fitness for duty"". ""fit for duty"" in accordance with the specifications.`

const result = parse(line, {
  skip_empty_lines: true,
  trim: true,
});

console.log(result)

Additional Context

I don't believe the relax_quotes option will fix this issue, because the quotes themselves are escaped.

wdavidw commented 2 months ago

The escape option apply to quoted fields. Either set escape: null or surround you field with quotes.

evanrh commented 2 months ago

The escape option apply to quoted fields. Either set escape: null or surround you field with quotes.

Setting escape: null doesn't work either. I tried also setting relax_quotes: true, but that doesn't work when there are quoted fields in the input data. We have both quoted and unquoted fields in the input, but it appears to only be quoting them when a newline is in the column.

I'll see about getting the data formatted properly, and I do see your point about it only being in quoted fields.