tuva-health / tuva

Main repo including core data model, data marts, reference data, terminology, and the clinical concept library
https://thetuvaproject.com/
192 stars 50 forks source link

safe_cast() appears not to map to try_cast() in Databricks #672

Open dr00b opened 6 days ago

dr00b commented 6 days ago

Describe the bug - Required DBT safe_cast macro fails on SQL Warehouse Pro in Databricks. Context in Slack

In service category staging code

{% if target.type == 'duckdb' %}
        where try_cast('hcpcs_code' as integer) is not null
{% else %}
        where {{ safe_cast('hcpcs_code', 'int')}} is not null
{% endif %}

DBT cross database macro appears not to be mapped to try_cast function (available in Spark SQL and Databricks SQL). I attempted revising to below, which still maps to cast(). {{ dbt.safe_cast("hcpcs_code", api.Column.translate_type("integer")) }}


Environment - Required


To Reproduce Steps to reproduce the behavior:

  1. Clone latest Tuva project
  2. Run locally against SQL Warehouse Pro, in Databricks

Expected behavior Code works when try_cast() is called.


Additional context Happy to address this ticket. Just need to test a revision w/ Databricks as target.type in profile.yml.