snowplow / dbt-snowplow-utils

Snowplow utility functions to be used in conjunction with the snowplow-web dbt package.
Other
13 stars 6 forks source link

Fixes + check for dupes contexts in postgres/redshift #139

Closed rlh1994 closed 1 year ago

rlh1994 commented 1 year ago

Description & motivation

@agnessnowplow Raised an issue around when the session/user identifiers have the same context it fails on PG/RS (due to duplicate CTEs with the same name). This is actually a wider issue, it would fail with the same context used multiple times in one of the identifiers, and also if you use the context as an identifier then add it in the entity_or_sde as well. Giving each CTE a custom name felt complicated, so I have instead made the logic directly on the base macros.

Basically, I have got a distinct list of contexts that need CTEing and joining, and where there are duplicates it uses the alias and prefix of the FIRST instance, but the FIELD of the actual record (please double check I haven't misses this anywhere). I do not raise a warning in the manifest because they should never need to access the inner workings of this code so we can just manage it for them.

I also made a few other tweaks while I was here.

We should probably add some tests for this, but that felt like a large task that we may not have time to do before release

Checklist