ironSource / parquetjs

fully asynchronous, pure JavaScript implementation of the Parquet file format
MIT License
349 stars 176 forks source link

Problem with materialize on an object with optional missing variable #39

Closed ZJONSSON closed 6 years ago

ZJONSSON commented 6 years ago

In some cases where a property is optional in the schema and missing from the record, other values will not be missing as well from materialize.

Example:

Actual output [ { fruit: {} } ] Expected output : [ { fruit: { name: 'apple' } } ]

const parquet = require('../parquetjs');

var schema = new parquet.ParquetSchema({
  fruit: {
    fields: {
      name: { type: 'UTF8' },
      colour: { type: 'UTF8', optional: true }
    }
  }
});

let buf = {};
let rec = { fruit: { name: 'apple'}};

parquet.ParquetShredder.shredRecord(schema, rec, buf);
let records = parquet.ParquetShredder.materializeRecords(schema, buf);

console.log(records);