tuva-health / tuva

Main repo including core data model, data marts, reference data, terminology, and the clinical concept library
https://thetuvaproject.com/
172 stars 41 forks source link

apply_regex DuckDB #536

Closed dprovder closed 1 month ago

dprovder commented 1 month ago

Running Tuva_demo with duck db gives a runtime error for hcc_suspecting models which use apply_regex

There doesn't seem to be a macro "apply_regex" for duckDB, & so when running hcc_suspecting models, a runtime error is produced.

Runtime Error in model hcc_suspecting__int_prep_egfr_labs (models/hcc_suspecting/intermediate/hcc_suspecting__int_prep_egfr_labs.sql)
  Catalog Error: Scalar Function with name regexp_like does not exist!
  Did you mean "regexp_replace"?
  LINE 79:     where regexp_like(result, '^[+-]?([0-9]*[.])?[0-9]+$')

I tried to directly modify the intermediate models which used the regex the macro, to no avail. Also tried disabling clinical which didn't seem to work either.

After adding these lines in macros/apply_regex.sql:

{%- macro duckdb__apply_regex(column_name, regex) -%}

    regexp_matches({{ column_name }}, '{{ regex }}')

{%- endmacro -%}

the models ran successfully.

sarah-tuva commented 1 month ago

Hello! Thanks for logging this issue. We were able to test it on our end as well. Are you willing to create a pull request with this fix so we can include it in the next package release?

sarah-tuva commented 1 month ago

Resolved with PR #544. Thanks for @dprovder! 🙏