search

ironSource / parquetjs

fully asynchronous, pure JavaScript implementation of the Parquet file format
MIT License
346 stars 175 forks source link

Cannot write parquet file with zero rows #81

Open donnut opened 5 years ago

donnut commented 5 years ago

I'm streaming data to the parquetTransformer and sometimes the stream does not contain any data. The transformer throws the error: Error: the string "cannot write parquet file with zero rows" was thrown, throw an Error :).

What is the best way to handle this situation? I don't want to check for empty streams in the application itself. I tested this situation by making mkTestRows return an empty array in test/integration.js

dogenius01 commented 4 years ago

Hi, @donnut

I'm facing similar issue. But, i cannot catch the rejection from parquetTransformer.

How did you write code to catch that error?

KingAMS commented 4 years ago

any Ide @dogenius01 / @donnut ?

flobjective commented 3 years ago

I am facing the same issue. As I am new to parquet (world) - is there a specific reason to disallow files with zero rows? Is it considered bad practice?

morokhovets commented 3 years ago

I am facing the same issue too. My use case is that I need to rotate files by time regardless if it contains any rows or not.

There is a commit in master where this issue is fixed. But there were no releases since then. https://github.com/ironSource/parquetjs/commit/9cc7b4d9937970be84811cf5564f64c940e7277e

Will use master or commit hash in package.json, I guess.