search

fivetran / dbt_fivetran_utils

Helper utils for our packages
29 stars 19 forks source link

Bugfix/snowflake json extract #50

Closed fivetran-joemarkiewicz closed 2 years ago

fivetran-joemarkiewicz commented 2 years ago

What change does this PR introduce?

This PR adds a new `Snowflake` designation to the `json_extract_path` macro. Included in this new designation is the addition of the `try_parse_json` function to check if the field is a json object. If it is not, then `null` is generated. If this is not added and a record is not a json object then the function will fail. See [this issue](https://github.com/fivetran/dbt_fivetran_log/issues/26) for details on why it fails. **If this PR introduces a new macro, how did you test the new macro?**

No, just an update to an existing macro.

If this PR introduces a modification to an existing macro, which packages is the macro currently present in and what steps were taken to test compatibility across packages?

This macro is used in a variety of packages. It has been tested on the dbt_fivetran_log package and comfortable it will succeed as intended in others as well.

Did you update the README to reflect the macro addition/modifications?

This is an under the hood update and a README update is not needed. However, I did update the CHANGELOG

fivetran-joemarkiewicz commented 2 years ago

I decided to test this on Postgres and Redshift as well. I found that Postgres worked fine when the field is not always a json object. However, Redshift failed similar to Snowflake. I updated the PR to account for this in the Redshift macro designation as well.