Open ernestoongaro opened 1 year ago
Thanks for reporting this @ernestoongaro !
I got the same results as you when I tried the same thing in dbt-duckdb.
There's two main options for resolving this:
I'm not sure if there's an easy path to support referencing ephemeral nodes from dbt python models or not. Lots of juicy details in the Compiler class of how ephemeral models are injected as a CTE into SQL models (and apparently Python models as well 😬 ).
If it's non-trivial to add support for Python, one idea would be to proactively raise as an error in the meantime to say that referencing ephemeral models isn't supported.
Maybe add something like this somewhere around here:
if node.language == ModelLanguage.python:
raise DbtInternalError(
f"During compilation, found an ephemeral model named {cte.id}, but ephemeral models are not supported for Python."
)
Of course we'd want to add an explanation within docs.getdbt.com as well if we choose Option 2.
Option 2 makes sense to me in the shorter term!
We do something similar within the run-operation
task today (which can't handle direct references to ephemeral models because of ... honestly, let's not get into it):
I don't think it would make sense to have a custom RefResolver
just for Python models, but we could add some conditional logic to RuntimeRefResolver
that checks for self.language
and target_model.is_ephemeral_model
.
PR to update the docs (thanks @ernestoongaro!!)
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.
Marking as un-stale because there's an open PR and it also has a recent comment: https://github.com/dbt-labs/dbt-core/pull/9196#issuecomment-1923322305
Is this a new bug in dbt-core?
Current Behavior
Doing this writes the SQL directly into the Python file and fails fairly miserably
Expected Behavior
Ideally it would be good support this, or give a compilation error vs trying to send invalid python down
Steps To Reproduce
select 1 as test
, save it asmy_ephemeral_sql_model.sql
Write this python file
Relevant log output
Environment
Which database adapter are you using with dbt?
snowflake
Additional Context
No response