kestra-io / kestra

:zap: Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 500+ plugins. Alternative to Zapier, Rundeck, Camunda, Airflow...
https://kestra.io
Apache License 2.0
11.67k stars 997 forks source link

The Running duration of each dbt node is no longer parsed in the Gantt view #4767

Closed AlxGlx closed 1 month ago

AlxGlx commented 2 months ago

Describe the issue

Gantt lines are not appearing correctly for DBT model builds. Run Duration in the hover over is also incorrect. image

in previous builds the gantt chard displays like this: image

Environment

anna-geller commented 2 months ago

do you have a reproducer? running duration of 0s seems suspicious indeed

anna-geller commented 2 months ago

ok, I was able to reproduce with the tutorial flow. Looks like the issue in parsing manifest.json. Thanks for reporting this!

id: dwh_and_analytics
namespace: tutorial
description: Data Warehouse and Analytics

tasks:
  - id: dbt
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
    - id: clone_repository
      type: io.kestra.plugin.git.Clone
      url: https://github.com/kestra-io/dbt-demo
      branch: main

    - id: dbt_build
      type: io.kestra.plugin.dbt.cli.DbtCLI
      parseRunResults: true
      taskRunner:
        type: io.kestra.plugin.scripts.runner.docker.Docker
      containerImage: ghcr.io/kestra-io/dbt-duckdb:latest
      commands:
        - dbt deps
        - dbt build
      profiles: |
        jaffle_shop:
          outputs:
            dev:
              type: duckdb
              path: dbt.duckdb
              extensions: 
                - parquet
              fixed_retries: 1
              threads: 16
              timeout_seconds: 300
          target: dev      
mgabelle commented 1 month ago

Created PR in plugin-dbt repo :PR_141