mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

`\r` and `\n` arriving in separate chunks breaks line separator detection #234

Open alecmev opened 9 months ago

alecmev commented 9 months ago

Expected Behavior

The assert in the repro passes.

Actual Behavior

The assert in the repro fails.

How Do We Reproduce?

import { deepStrictEqual } from 'node:assert';
import { Readable } from 'node:stream';

import csvParserFactory from 'csv-parser';

deepStrictEqual(
  await Readable.from(["foo\r", "\nbar\r\n", "baz\r\n"])
    .pipe(csvParserFactory())
    .toArray(),
  await Readable.from(["foo\r\n", "bar\r\n", "baz\r\n"])
    .pipe(csvParserFactory())
    .toArray(),
);

Cause

This happens because csv-parser is fine with nextChr being null when this.state.first is true. Instead it should wait until there's at least one more byte or the input stream is closed.