search

gojekfarm / beast

[Deprecated] Load data from Kafka to any data warehouse. BQ sink is being supported in Firehose now. https://github.com/odpf/firehose
Apache License 2.0
148 stars 23 forks source link

Feature: namespace BQ metadata field when provided #64

Closed rootcss closed 4 years ago

rootcss commented 4 years ago

when BQ_METADATA_NAMESPACE= config is set to some value, namespace the kafka offset metadata in BQ i.e, as a RECORD/STRUCT type.

By default, the value will be empty, which means metadata fields will be written as different columns rather than a RECORD/STRUCT type.

rootcss commented 4 years ago

@sravankorumilli Please review and merge.

mauliksoneji commented 4 years ago

@rootcss can you please rebase against master