search

dbt-labs / spark-utils

Utility functions for dbt projects running on Spark
https://hub.getdbt.com/fishtown-analytics/spark_utils/latest/
Apache License 2.0
30 stars 15 forks source link

Regexp_intr shim support for dbt-expectations macro #29

Open fivetran-joemarkiewicz opened 1 year ago

fivetran-joemarkiewicz commented 1 year ago

Since dbt-expectations removed support for the Databricks and Spark macro compatibility of their macros, I believe it would be best to move some support of these macros into this package.

I currently have a working version of the regexp_instr macro for spark that is outlined within the dbt-expectations PR #226. Claus makes a great point to instead move this PR into this project. Before moving forward with opening a PR I wanted to confirm this approach is the desired way forward within this Issue. Specifically, I noticed the last release was almost a year ago. Would you prefer to continue adding to this project, or is this a shim we should instead provide for our packages within fivetran_utils?

Let me know what you feel the best path forward would be. Thanks!

followingell commented 1 year ago

Following since I raised a similar issue in dbt-expectations: https://github.com/calogica/dbt-expectations/issues/228.

fivetran-joemarkiewicz commented 1 year ago

I ended up having some available time today and opened the above PR to incorporate dbt-expectations dispatches. I only added a shim for regexp_instr but added the folder so more may be added in the future if needed.