snowplow / snowbridge

For replicating streams across clouds, accounts and regions
Other
14 stars 8 forks source link

Remove `null` fields from JQ transformation output #357

Closed pondzix closed 1 month ago

pondzix commented 1 month ago

PDP-1295

We don't want explicit null to be present on JQ output, so we have to get rid of them somehow. As there is no option neither in gojq nor json.Marshal allowing us to to that, it seems the best way is to simply iterate over a map (we have a map between gojq output and json.Marshal input in transformation) and filter out fields being nulls.