mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

csv-parser fails when given a stream from `Stream` instead of `fs` #213

Open venya02 opened 2 years ago

venya02 commented 2 years ago

Expected Behavior

Creating a Readable stream and piping it into the parser should result in parsed csv data

Actual Behavior

internal/streams/readable.js:623
  throw new ERR_METHOD_NOT_IMPLEMENTED('_read()');
  ^

Error [ERR_METHOD_NOT_IMPLEMENTED]: The _read() method is not implemented
    at Readable._read (internal/streams/readable.js:623:9)
    at Readable.read (internal/streams/readable.js:462:10)
    at maybeReadMore_ (internal/streams/readable.js:610:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  code: 'ERR_METHOD_NOT_IMPLEMENTED'
}

How Do We Reproduce?

const Stream = require('stream')
const readableStream = new Stream.Readable()
const csv = require('csv-parser')
const fs = require("fs")
const results = [];

fs.createReadStream('csvTest.csv')
    .pipe(csv())
    .on('data', (data) => results.push(data))
    .on('end', () => {
    console.log(results);
    // [
    //   { NAME: 'Daffy Duck', AGE: '24' },
    //   { NAME: 'Bugs Bunny', AGE: '22' }
    // ]
    });

readableStream.push(`example,csv,data
    line,two,is
    not,needed,but
    I,made,one
    anyway,0,1`)

readableStream.pipe(csv())
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
    // [
    //   { NAME: 'Daffy Duck', AGE: '24' },
    //   { NAME: 'Bugs Bunny', AGE: '22' }
    // ]
  });

csvTest.csv

both methods of creating a stream should result in the exact same output, as they have the exact same input.

Kemosabert commented 1 year ago

Not sure if you still have the issue, but after pushing your actual data, you should also push null to the readable stream like such: readableStream.push(null).