dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.96k stars 1.63k forks source link

Track batch execution time for microbatch models #10828

Closed QMalcolm closed 1 month ago

QMalcolm commented 1 month ago

Resolves #10825

Problem

We weren't tracking the execution time of batches for microbatch models. Thus, we were always logging that they took no time to run (0.00 seconds) which didn't actually match reality.

Solution

Begin tracking the time it takes for a batch to run, and propagate that information to the batch run result.

Checklist

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 89.13%. Comparing base (6b9c1da) to head (b9f7332). Report is 4 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #10828 +/- ## ========================================== - Coverage 89.20% 89.13% -0.07% ========================================== Files 183 183 Lines 23402 23418 +16 ========================================== - Hits 20875 20874 -1 - Misses 2527 2544 +17 ``` | [Flag](https://app.codecov.io/gh/dbt-labs/dbt-core/pull/10828/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs) | Coverage Δ | | |---|---|---| | [integration](https://app.codecov.io/gh/dbt-labs/dbt-core/pull/10828/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs) | `86.36% <100.00%> (-0.15%)` | :arrow_down: | | [unit](https://app.codecov.io/gh/dbt-labs/dbt-core/pull/10828/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs) | `62.11% <50.00%> (-0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs#carryforward-flags-in-the-pull-request-comment) to find out more. | [Components](https://app.codecov.io/gh/dbt-labs/dbt-core/pull/10828/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs) | Coverage Δ | | |---|---|---| | [Unit Tests](https://app.codecov.io/gh/dbt-labs/dbt-core/pull/10828/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs) | `62.11% <50.00%> (-0.01%)` | :arrow_down: | | [Integration Tests](https://app.codecov.io/gh/dbt-labs/dbt-core/pull/10828/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dbt-labs) | `86.36% <100.00%> (-0.15%)` | :arrow_down: |