mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

What would be the vanilla version of the pipe() process. #192

Closed StephaneAdeso closed 3 years ago

StephaneAdeso commented 3 years ago

In the documentation, all examples are with the "pipe()" method. Can someone tell me the vanilla version of the following example:

const csv = require('csv-parser')
const fs = require('fs')
const results = [];

fs.createReadStream('data.csv')
  .pipe(csv())
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
    // [
    //   { NAME: 'Daffy Duck', AGE: '24' },
    //   { NAME: 'Bugs Bunny', AGE: '22' }
    // ]
  });

I do not know if I explained well. I am new to node and javascript. And although I understand how the "pipe ()" method works, I need to perform that operation manually. If you can add that example to the documentation by the way, it would be great for novice programmers like me.

TrySound commented 3 years ago

By vanilla you mean browser version? You need to configure alias in your bundler to this package https://github.com/nodejs/readable-stream and include buffer polyfill. Browser streams will be eventually implemented in node which will let us to make csv-parser more universal.

StephaneAdeso commented 3 years ago

Hello, thanks for responding so quickly. I explain myself a little better. I am creating an application ( Node + Electron project) that will take local csv files, save that data in javascript objects to be able to apply mathematical formulas to them and at the end of all, save them to local files. The problem with the "pipe ()" is that it is a very strong and narrow flow. That doesn't suit me. I need to be able to control when to save the data, when to go to the next file etc ... If I use "pipe ()", it forces me to start and finish on the fly. And I can't use "async / await" either because "fs.createReadStream ()" doesn't return a promise to me. So the question would be how can I do the same as with the "pipe ()" but manually. Or better yet, how can I use "csv-parser" but instead of using a local file, I already have a "string". So the idea would be to be able to store each step of the process in variables to be able to use them as needed instead of using the "pipe ()".

TrySound commented 3 years ago

You can just wrap stream with promise. Not perfect, I agree, we will come to better solution some day.

  const result = await new Promise((resolve, reject) => {
    const array = [];
    fs.createReadStream('./contacts.csv')
      .on('error', error => reject(error))
      .pipe(csv())
      .on('data', object => array.push(object))
      .on('error', error => reject(error))
      .on('end', () => resolve(array));
  });
StephaneAdeso commented 3 years ago

thanks i will try this