Open hbaldwinRPT opened 3 years ago
Which docs did you use (link)? And which specific version of the lib (we have an rc out for v6)?
Thanks for the quick response, sorry for my ignorance haha.
The docs: https://www.npmjs.com/package/json2csv version 5.0.6 not 6RC
I can go ahead and try that version if that would help possibly.
use the .transform
property of the AsyncParser.processor
, it will give you a transform stream, which can be used as a read stream.
That's how I managed to directly stream into a csv file inside an s3 bucket
v6, which has moved to a new repo and broken down has rewritten the async api to be much simpler.
Full docs here: https://juanjodiaz.github.io/json2csv/#/parsers/node-async-parser
Although just realizing it doesn't says it: parser.parse(data)
returns the transform stream so you can do parser.parse(data).pipe(s3BucketStream)
The docs mention that AsyncParser.processor is a readable stream, but when i utilize it it says that it is a writeable stream and therefore i am having trouble piping the stream to s3 using the upload function that requires a readable stream.
Can someone help me out?
node version is 12+ latest version of the package
there are no errors aor anything, just trying to figure out how to get a readable stream. i cannot pipe to the s3 using output that i know of since the s3.upload is not a readable stream, just takes a readable stream.
any help is super appreciated.