Hey @mjirv, thanks for this incredibly useful package!
This PR is intended to fix a bug where fields are being included in the comparison even if they only match part of a field name in compare_columns. For example, if a table has the columns foo and is_foo_active, and compare_columns is ["is_foo_active"] (without foo), foo gets included anyway.
I’m new to Jinja and Python, but it looks to me like running compare_columns through upper() is converting it to a string, on which the in operator performs a substring search instead of the intended element-by-element list search. This PR uses map() to uppercase each element of compare_columns individually, preserving it as a list, I think.
I haven’t added an integration test, but I’d be happy to if you think it’s warranted, @mjirv.
Checklist
[ ] I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
[x] BigQuery
[ ] Postgres
[ ] Redshift
[ ] Snowflake
[ ] I have updated the README.md (if applicable)
[ ] I have added tests & descriptions to my macros (and models if applicable)
This is a:
Description & motivation
Hey @mjirv, thanks for this incredibly useful package!
This PR is intended to fix a bug where fields are being included in the comparison even if they only match part of a field name in
compare_columns
. For example, if a table has the columnsfoo
andis_foo_active
, andcompare_columns
is["is_foo_active"]
(withoutfoo
),foo
gets included anyway.I’m new to Jinja and Python, but it looks to me like running
compare_columns
throughupper()
is converting it to a string, on which thein
operator performs a substring search instead of the intended element-by-element list search. This PR usesmap()
to uppercase each element ofcompare_columns
individually, preserving it as a list, I think.I haven’t added an integration test, but I’d be happy to if you think it’s warranted, @mjirv.
Checklist