search

datahub-project / datahub

The Metadata Platform for your Data and AI Stack
https://datahubproject.io
Apache License 2.0
9.93k stars 2.94k forks source link

LookML lineage to bigquery is not shown correctly due to issues in resolving variables #11893

Open claudiovalerio5005 opened 2 days ago

claudiovalerio5005 commented 2 days ago

Describe the bug We are running a looked ingestion, using datahub 14.1. Our database is bigquery. we have a problem on resolving lineage for lookml view.

the view is defiend has below:

# The name of this view in Looker is "Dm Sales Flash"
view: dm__sales_flash {
  # The sql_table_name parameter indicates the underlying database table
  # to be used for all fields in this view.
  sql_table_name: @{ric_dp_finance_sa_schema}.bqdts_controlling_copa_dm.dm__sales_flash ;;

as you can see the sql_table_name statemetn contains a variable @{ric_dp_finance_sa_schema}.

we are trying to map the variable in the lookML ingestion with setting:

    liquid_variable:
        ric_dp_finance_sa_schema: "bigquery_project_id"

Unfortunately that does not work, the lineage is not resolved.

image

In adition, i we try to assign manuallythe lineage, it will get removed the next timee the lookml ingestion runs.