Global concurrency_key limits do not affect asset materialization

[VerdantForge]

Dagster version

1.7.10

What's the issue?

Op level concurrency limits set globally through the dagster.yaml file don't seem to be working at all. Example from our dagster.yaml

run_coordinator:
  module: dagster.core.run_coordinator
  class: QueuedRunCoordinator
  config:
    max_concurrent_runs: 5
    tag_concurrency_limits:
      - key: "dagster/concurrency_key"
        value: "libretranslate"
        limit: 2

however creating the same configuration at the level of the asset job works just fine

asset_job = define_asset_job(
    name="asset_job",
    description="Provides scheduling to asset job",
    selection=[
        asset_that_we_have.key
    ],
    config={
        "execution": {
            "config": {
                "multiprocess": {
                    "tag_concurrency_limits": [
                        {
                            "key": "dagster/concurrency_key",
                            "value": "libretranslate",
                            "limit": 2,
                        }
                    ],
                },
            }
        }
    },
)

What did you expect to happen?

I expected the only 2 ops to be launched accross the entire system for any number of runs that might be launched in parallel. However this was not the case, even for a single run with a dynamic op which was supposed to be throttled, it launched many in parallel.

How to reproduce?

No response

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[divyanshu-in]

@garethbrickman looking into it

[VerdantForge]

Any progress since last week? Is there some info I can give you?

[divyanshu-in]

I got little busy with some personal stuff, but I started with storing the op/asset tags to the storage so that later on it can be compared in concurrency logic, can I take some more time, I assure you it will be done by end of this week.