LibertyDSNP / parquetjs

Fully asynchronous, pure JavaScript implementation of the Parquet file format with additional features
MIT License
43 stars 24 forks source link

`ParquetWriter.createListField()` creates broken schema #132

Open kmhleonwillens opened 5 days ago

kmhleonwillens commented 5 days ago

Thanks for reporting an issue!

Steps to reproduce

Create a schema with the ParquetWriter utility and describe a primitive array field, and try to append a row with a matching input:

const testSchema = new ParquetSchema({
  doesWork: ParquetWriter.createStringField(),
  doesNotWork: ParquetWriter.createListField('UTF8'),
});

const writer = await ParquetWriter.openFile(testSchema, '/tmp.parquet');
await writer.appendRow({ doesWork: 'foo', doesNotwork: ['bar', 'baz'] });
await writer.close();

Expected behaviour

The writer should accept the row input as it seemingly matches the schema definition.

Actual behaviour

The row is not appended as the writer throws an error.

Any logs, error output, etc?

When catching the error, the output is:

too many values for field: doesNotWork

Any other comments?

It works when defining the list field manually, as seen in the README file.