snowplow / iglu-central

Contains all JSON Schemas, Avros and Thrifts for Iglu Central
http://iglucentral.com
Apache License 2.0
118 stars 114 forks source link

Common: remove polymorphic fields from bad rows schemas #1075

Open istreeter opened 4 years ago

istreeter commented 4 years ago

Some of the bad rows schemas contain "polymorphic" fields, e.g. the field value can be an object or a string, depending on the context. This causes problems for querying the badrows in BigQuery, where fields must have a single well-defined type.

The schemas should be simplified so that each field has only one type.

Here is a list of the affected fields:

istreeter commented 4 years ago

See relevant ticket in badrows