coderxio / sagerx

Open drug data pipelines curated by pharmacists.
https://coderx.io/sagerx
Other
41 stars 11 forks source link

Convert Postgres-specific SQL in dbt models to BigQuery-specific SQL where needed #286

Open jrlegrand opened 1 month ago

jrlegrand commented 1 month ago

Problem Statement

Moving from PostgreSQL to BigQuery means we will definitely find SQL that throws errors d/t the different systems.

We need to identify these areas and modify the SQL to match BigQuery and then TEST to make sure it doesn't break existing marts, etc.

Criteria for Success

All dbt models run in BigQuery.

Run whatever the command is for a full dbt refresh and monitor for errors.

Additional Information

One way we know BigQuery is significantly different is how it handles dates: https://cloud.google.com/bigquery/docs/reference/standard-sql/date_functions