Add task timeout support.

t0momi219 commented 1 week ago

Description

In Airflow, both DAGs and tasks can have timeout specified. Since dbt models likely have varying expected execution times for each layer, there could be cases where users want to apply timeout individually to each node.

How about having Cosmos retrieve timeout from the node metadata and apply them individually when rendering nodes?

Specifically, the expected time is specified in the model's config, which will be read accordingly.

version: 2
models:
  - name: stg_customers
    config:
      cosmos_task_timeout: 600 # Specify in seconds.

Related Issue(s)

closes https://github.com/astronomer/astronomer-cosmos/issues/1316

Breaking Change?

Checklist

[x] I have made corresponding changes to the documentation (if required)
[x] I have added tests that prove my fix is effective or that my feature works

t0momi219 commented 4 days ago

Hi @pankajkoti , This PR is ready for review. Could you please check this?

codecov[bot] commented 4 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.85%. Comparing base (a5de8b4) to head (4bd843a). Report is 7 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #1317 +/- ## ======================================= Coverage 95.85% 95.85% ======================================= Files 67 67 Lines 3979 3982 +3 ======================================= + Hits 3814 3817 +3 Misses 165 165 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

astronomer / astronomer-cosmos