Convert csv stream to UTF8?

[matbz]

Hi, I have a csv stream:

const csvReadStream = fs.createReadStream(file.path);

which I need to convert to utf8. Can I do this with this lib? If so, I would appreciate some pointers. Thanks!

[bnoordhuis]

Yes, you can. Did you check the README? Is anything unclear?

[matbz]

Yes and I've found this:

// Convert encoding streaming example fs.createReadStream('file-in-win1251.txt') .pipe(iconv.decodeStream('win1251')) .pipe(iconv.encodeStream('ucs2')) .pipe(fs.createWriteStream('file-in-ucs2.txt'));

But I'm not sure how I can use this to transfer my coding so that in the end "csvReadStream" is the content of the file in file.path encoded to UTF8. I'm pretty new to node.js and programming in general so I'm really sorry for this basic question.

[bnoordhuis]

That's the iconv-lite API, different module. :-)

With iconv, you'd do this:

const { Iconv } = require('iconv');
fs.createReadStream(new Iconv('cp1251', 'utf8')).pipe(sink)

[matbz]

Oh :)

Instead of 'cp1251' I would put file.path in my example right? But what is "sink"? Sorry again for this basic question but I really appreciate your help!

[bnoordhuis]

Oh, wups - sorry, I mistyped. I mean this:

fs.createReadStream(filename).pipe(new Iconv('cp1251', 'utf8')).pipe(sink)

sink would be wherever you want to send the data to - the WriteStream in your example.

[matbz]

I tried

const csvReadStream = fs.createReadStream(file.path).pipe(new Iconv('iso-8859-1', 'utf8')).pipe(file.path + '_new');

but it doesn't work. Am I missing something / do I use it incorrectly?

[bnoordhuis]

You always pipe to a stream. A string is not a stream, you probably were looking for fs.createWriteStream(file.path + '_new').

[matbz]

Now I get it. Actually used iconv-lite now and works perfectly. Thank you so much!