Open donnut opened 5 years ago
Hi, @donnut
I'm facing similar issue. But, i cannot catch the rejection from parquetTransformer.
How did you write code to catch that error?
any Ide @dogenius01 / @donnut ?
I am facing the same issue. As I am new to parquet (world) - is there a specific reason to disallow files with zero rows? Is it considered bad practice?
I am facing the same issue too. My use case is that I need to rotate files by time regardless if it contains any rows or not.
There is a commit in master where this issue is fixed. But there were no releases since then. https://github.com/ironSource/parquetjs/commit/9cc7b4d9937970be84811cf5564f64c940e7277e
Will use master or commit hash in package.json, I guess.
I'm streaming data to the
parquetTransformer
and sometimes the stream does not contain any data. The transformer throws the error:Error: the string "cannot write parquet file with zero rows" was thrown, throw an Error :)
.What is the best way to handle this situation? I don't want to check for empty streams in the application itself. I tested this situation by making
mkTestRows
return an empty array intest/integration.js