search

fal-ai / dbt-fal

do more with dbt. dbt-fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.
https://fal.ai/dbt-fal
Apache License 2.0
853 stars 72 forks source link

feat(dbt-fal): adapt to new 1.5 dbt changes #814

Closed chamini2 closed 1 year ago

chamini2 commented 1 year ago

Description

Refactor and adapt DB profile loading for dbt-fal. It mostly follows the same ideas adapted to some changes on how flags are loaded and avoiding a new parser for some checks. In general this is a better implementation than before but does not really take advantage of any new API introduced in 1.5

Other good thing with this change is that now running dbt run on the integration test projects does not need a temp_dir env var being set, becuase it handles it exclusively for behave.

Postgres

Got it to just fail for features/source.feature:9 Run a Python model that queries a source in local environment. That failure would be related to fal-cli tool, because we fill the source with a FalDbt instance.

❯ python load_freshness_table.py . ~/.dbt/
Traceback (most recent call last):
  File "load_freshness_table.py", line 3, in <module>
    from fal import FalDbt
  File "/Users/matteo/.pyenv/versions/fal15/lib/python3.8/site-packages/fal/__init__.py", line 3, in <module>
    from faldbt.project import (
  File "/Users/matteo/.pyenv/versions/fal15/lib/python3.8/site-packages/faldbt/project.py", line 20, in <module>
    from dbt.contracts.graph.parsed import (
ModuleNotFoundError: No module named 'dbt.contracts.graph.parsed'

Snowflake

BigQuery

Redshift

Changed connection solution, will be addressed in PR #835

linear[bot] commented 1 year ago

FEA-932 Support dbt 1.5 for dbt-fal