snowplow / snowbridge

For replicating streams across clouds, accounts and regions
Other
15 stars 7 forks source link

Remove `null` fields from JQ transformation output #357

Closed pondzix closed 3 months ago

pondzix commented 4 months ago

PDP-1295

We don't want explicit null to be present on JQ output, so we have to get rid of them somehow. As there is no option neither in gojq nor json.Marshal allowing us to to that, it seems the best way is to simply iterate over a map (we have a map between gojq output and json.Marshal input in transformation) and filter out fields being nulls.