BurntSushi / xsv

A fast CSV command line toolkit written in Rust.
The Unlicense
10.35k stars 321 forks source link

xsv fixlengths never finishes on some input #312

Open seamusabshere opened 1 year ago

seamusabshere commented 1 year ago
$ xsv stats bad.csv
CSV error: record 1501690 (line: 1501690, byte: 419783925): found record with 39 fields, but the previous record has 38 fields

$ csvclean bad.csv > fixed.csv
CSV contains fields longer than maximum length of 131072 characters. Try raising the maximum with the field_size_limit parameter, or try setting quoting=csv.QUOTE_NONE.

$ xsv fixlengths bad.csv > fixed.csv
^C

xsv fixlengths never finishes

(csvclean is from https://csvkit.readthedocs.io/en/1.1.0/scripts/csvclean.html)

BurntSushi commented 1 year ago

Well there's not much I can do about the bug if you don't share the input that reproduces the problem...