(pinging @OwenKephart, this is the issue as you suggested).
What did you expect to happen?
Self-dependent time partitioned assets automaterialize correctly (in my view) when they are root assets. They automaterialize sequentially (partition by partition) and the most recent materializations_per_minute partitions are materialized. That is:
if asset is defined with AutomaterializePolicy.eager(1) only the last partition is materialized
if asset is defined with AutomaterializePolicy.eager(3) (as in repro) three most recent partitions are materialized
if asset is defined with AutomaterializePolicy.eager(∞), all partitions are materialized.
I expect the same to hold when self-dependent time partitioned assets descend from unpartitioned asset.
Dagster version
1.4.16
What's the issue?
As stated in the title.
(pinging @OwenKephart, this is the issue as you suggested).
What did you expect to happen?
Self-dependent time partitioned assets automaterialize correctly (in my view) when they are root assets. They automaterialize sequentially (partition by partition) and the most recent
materializations_per_minute
partitions are materialized. That is:AutomaterializePolicy.eager(1)
only the last partition is materializedAutomaterializePolicy.eager(3)
(as in repro) three most recent partitions are materializedAutomaterializePolicy.eager(∞)
, all partitions are materialized.I expect the same to hold when self-dependent time partitioned assets descend from unpartitioned asset.
How to reproduce?
repro here: https://github.com/psarka/repro/tree/auto-time-window-2
Deployment type
None
Deployment details
No response
Additional information
Supersedes https://github.com/dagster-io/dagster/pull/16646 Supersedes https://github.com/dagster-io/dagster/issues/16033
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.