Pipe parse() and stringify()

[thom4parisot]

Hello there,

Thank you very much for this module :-)

I found it was unclear to me how to pipe read and write altogether – a type error (First argument must be a string, Buffer, ArrayBuffer, Array, or array-like object.) is reported when I try to achieve the following snippet:

fs.createReadStream(file)
        .pipe(JSONStream.parse('*'))
        .pipe(EventStream.mapSync(({ idVisit, actionDetails, serverDate }) => {
            // ..
            var visitClean = {
                idVisit,
                actions,
                date: serverDate,
            };

            return visitClean;
        })
       .pipe(writer)
       .pipe(JSONStream.stringify())
       .pipe(fs.createWriteStream(target_dir + path.basename(file)))

Any idea how to emit each object in an array in a streaming way?

Thanks a bunch :-)

[dominictarr]

if you removed .pipe(writer) your code would work. JSONStream.parse outputs like js objects, but something expects strings or buffers, so you need to put JSONStream.stringify() first.

[thom4parisot]

@dominictarr oh what a fool am I: I did not notice the writer remained in the chain.

const writer = JSONStream.stringify()
       .pipe(fs.createWriteStream(target_dir + path.basename(file)));

I moved things around and your solution clearly worked. Thanks a bunch :-) I am not yet sure why having stringify and fs.createWriteStream combined as writer did not work whereas when they are piped after mapSync it's alright but I will try to figure it out :-)

Thanks for your help again, love your work!

[dominictarr]

what did the code to writer look like?