search

sssuhanov / aod_downloader

0 stars 0 forks source link

dbt history incremental doesn't work #39

Closed sssuhanov closed 3 months ago

sssuhanov commented 3 months ago 
dagster_dbt.errors.DagsterDbtCliRuntimeError: The dbt CLI process with command`dbt build --select aod.marts.market_volume aod.staging.stg_aod__history aod.marts.total_volume aod.intermediate.int_aod__history_noduplicates aod.intermediate.int_aod__history`failed with exit code `1`. Check the stdout in the Dagster compute logs for the full information about the error, or view the dbt debug log: /opt/dagster/app/dbt/target/dbt_project_assets-dc61f14-27c71eb/dbt.log.Errors parsed from dbt logs:2 of 5 ERROR creating sql incremental model dbt_dev.int_aod__history ........... [ERROR in 0.18s]  Database Error in model int_aod__history (models/intermediate/int_aod__history.sql)  column "drop_id" does not exist  LINE 34: where h.id >= (select coalesce(max(drop_id), 0) from "postgr...                                              ^  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_plan.py", line 247, in dagster_event_sequence_for_step    for step_event in check.generator(step_events):  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 500, in core_dagster_event_sequence_for_step    for user_event in _step_output_error_checked_user_event_sequence(  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 184, in _step_output_error_checked_user_event_sequence    for user_event in user_event_sequence:  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 88, in _process_asset_results_to_events    for user_event in user_event_sequence:  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute.py", line 198, in execute_core_compute    for step_output in _yield_compute_results(step_context, inputs, compute_fn, compute_context):  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute.py", line 167, in _yield_compute_results    for event in iterate_with_context(  File "/usr/local/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 474, in iterate_with_context    next_output = next(iterator)  File "/opt/dagster/app/module/assets/dbt/__init__.py", line 17, in dbt_project_assets    yield from dbt.cli(["build"], context=context).stream()  File "/usr/local/lib/python3.10/site-packages/dagster_dbt/core/dbt_event_iterator.py", line 203, in __next__    return next(self._inner_iterator)  File "/usr/local/lib/python3.10/site-packages/dagster_dbt/core/dbt_cli_invocation.py", line 269, in _stream_asset_events    for event in self.stream_raw_events():  File "/usr/local/lib/python3.10/site-packages/dagster_dbt/core/dbt_cli_invocation.py", line 362, in stream_raw_events    self._raise_on_error()  File "/usr/local/lib/python3.10/site-packages/dagster_dbt/core/dbt_cli_invocation.py", line 454, in _raise_on_error    raise error
sssuhanov commented 3 months ago

maybe an answer: https://github.com/dagster-io/dagster/discussions/18880