mholt / PapaParse

Fast and powerful CSV (delimited text) parser that gracefully handles large files and malformed input
http://PapaParse.com
MIT License
12.38k stars 1.14k forks source link

Nodejs stream - support object mode #652

Open dsbert opened 5 years ago

dsbert commented 5 years ago

Upon further research, https://github.com/mholt/PapaParse/issues/568 indicates unparse does not support streaming at all.

The documentation for Nodejs stream support seems a bit limited and I'm not able to get it working in object mode. I'm getting an error when I attempt the following. Is object mode streaming supported?

Unable to serialize unrecognized input

const readable = new Readable({ objectMode: true, highWaterMark: 10000 })

// ...

const parser = Papa.unparse(Papa.NODE_STREAM_INPUT, {
  fastMode: true
});

pipeline(
  readable,
  through2.obj(function p(chunk, enc, cb) {
  // do stuff with data
  // chunk is a js object
    cb(null, chunk);
  }),
  parser,
  res
);
pokoli commented 5 years ago

Any pull request improving nodejs stream support is very welcome.