search

NYCPlanning / data-engineering

Primary repository for NYC DCP's Data Engineering team
14 stars 0 forks source link

Compile python requirements #828

Closed fvankrieken closed 1 week ago

fvankrieken commented 1 week ago

Automated changes by create-pull-request GitHub action

damonmcc commented 1 week ago

sqlfluff lint products/green_fast_track/models --templater=dbt failed with:

File "/usr/local/lib/python3.11/site-packages/sqlfluff_templater_dbt/templater.py", line 149, in dbt_config
  from dbt.config import read_user_config
  ^^^^^^^^^^^^^^^^^
ImportError: cannot import name 'read_user_config' from 'dbt.config' (/usr/local/lib/python3.11/site-packages/dbt/config/__init__.py)

may have to modify our sqlfluff fork so that our version of sqlfluff-templater-dbt can catch up to dbt-core==1.7.14

the latest sqlfluff-templater-dbt is 3.0.5 and we're using 2.3.1

fvankrieken commented 1 week ago

Ooh okay. I'll see if I can sort this out

fvankrieken commented 1 week ago

Weird though - I feel like this dependency issue should have been caught. As in, with our pinned forked sqlfluff, dbt-core should be pinned too and shouldn't have bumped. Maybe my sqlfluff fork somehow isn't pinned correctly to dbt-core? If that's handled outside of the repo, in pypi publishing or something.

Or really more importantly, a patch of dbt-core ideally shouldn't have breaking changes... This is all weird. Anyways, updating our sqlfluff fork to 3.0.5

fvankrieken commented 1 week ago

Actually, going to close this and just do this in my PR instead