Open VerdantForge opened 1 month ago
@garethbrickman looking into it
Any progress since last week? Is there some info I can give you?
I got little busy with some personal stuff, but I started with storing the op/asset tags to the storage so that later on it can be compared in concurrency logic, can I take some more time, I assure you it will be done by end of this week.
Dagster version
1.7.10
What's the issue?
Op level concurrency limits set globally through the
dagster.yaml
file don't seem to be working at all. Example from ourdagster.yaml
however creating the same configuration at the level of the asset job works just fine
What did you expect to happen?
I expected the only 2 ops to be launched accross the entire system for any number of runs that might be launched in parallel. However this was not the case, even for a single run with a dynamic op which was supposed to be throttled, it launched many in parallel.
How to reproduce?
No response
Deployment type
None
Deployment details
No response
Additional information
No response
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.