dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.2k stars 1.41k forks source link

Tags aren't applied on DBT assets #21206

Closed arookieds closed 5 months ago

arookieds commented 5 months ago

Dagster version

1.6.2 and 1.5.6

What's the issue?

Tags aren't applied on DBT assets, even though a dbt translator, with an overwritten get_tags is used

What did you expect to happen?

Tags to be applied on dbt assets

I have tried something in those lines:

{"maximum_concurrency_tag": "key_tag"}

How to reproduce?

DBT asset translator code:

def dbt_translator_factory(partition_mappings: dict, tags: Mapping[str, Any]):
    class DbtCustomTranslator(DagsterDbtTranslator):
        @classmethod
        def get_partition_mapping(
            cls,
            dbt_resource_props: Mapping[str, Any],
            dbt_parent_resource_props: Mapping[str, Any],
        ) -> Optional[PartitionMapping]:
            dbt_unique_id = dbt_resource_props["unique_id"]
            dbt_parent_unique_id = dbt_parent_resource_props["unique_id"]
            dbt_dependency = (dbt_unique_id, dbt_parent_unique_id)

            return partition_mappings.get(dbt_dependency)

        def get_tags(self, dbt_resource_props: Mapping[str, Any]) -> Mapping[str, str]:
            dbt_tags = dbt_resource_props.get("tags", [])
            print(f"DBT tags: {dbt_tags}")
            dagster_tags = {}
            for tag in dbt_tags:
                key, _, value = tag.partition("=")

                dagster_tags[key] = value if value else "__dagster_no_value"

            return {**dagster_tags, **tags}

    return DbtCustomTranslator

Method get_tags is never called during asset materialization.

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

arookieds commented 5 months ago

Dbt translator method get_tags was release in version 1.6.4.