dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.99k stars 1.63k forks source link

[CT-2111] [Feature] Allow `exposures` to depend on other `exposures` #6963

Closed david-beallor closed 1 year ago

david-beallor commented 1 year ago

Is this your first time submitting a feature request?

Describe the feature

Allow exposures to list other exposures in their list of dependencies. At Compass Digital, we have exposures representing Looker explores, and exposures representing Looker dashboards. It would be great if we could have the Looker dashboard exposures list the Looker explore exposures as it's dependencies.

Describe alternatives you've considered

Currently, we manually collate the list of dependencies from each explore exposure and list those as the dashboard dependencies.

Who will this benefit?

Anyone that has exposures that depend on other exposures :)

Are you interested in contributing this feature?

Yes, would love to contribute, but would require some guidance on the moving pieces here to get started.

Anything else?

No response

jtcohen6 commented 1 year ago

@david-beallor Thanks for opening!

This was discussed way back when:

Exposures representing Looker explores, and exposures representing Looker dashboards. It would be great if we could have the Looker dashboard exposures list the Looker explore exposures as it's dependencies.

This is indeed the use case:

I've been open to this idea in the past. Nowadays, I'm hesitant to make exposures anything other than a terminal/leaf node (always a child, never a parent) — in the same way that sources should only ever be a root node (always a parent, never a child).

Is there another way to go here, that looks like "grouping" exposures, similar to how we enable grouping common sources?

exposures:
  - name: looker_explore
    owner: ...
    maturity: high
    depends_on:
      - ref('my_model')
    nested_exposures: # this is a bad name, but you get the idea
      - name: specific_looker_dashboard
        # inherits properties from above, but could also override some
        maturity: low

It wouldn't be possible to have a "child" exposure referencing two "parent" exposures. Rather, it's really just a way to logically group related exposures together.

What do you think?

david-beallor commented 1 year ago

Thanks for the thorough response @jtcohen6!

I like the logic of keeping exposures as terminal nodes, but it would be great if there was a way to respect this, while also enabling "child" exposures to have multiple "parent" exposures. We often have Looker dashboards that have visuals from multiple explores, so the multiple "parent" functionality would be very useful.

What if the "child" to "parent" relationship was one of extension rather than nesting?

I'm thinking something like the following...

exposures:
  - name: looker_explore_orders
    owner: ...
    maturity: high
    depends_on:
      - ref('orders')
  - name: looker_explore_customers
    owner: ...
    maturity: high
    depends_on:
      - ref('customers')
  - name: looker_customer_orders_dashboard
    # Inherits unioned dependencies from the extended exposures
    extends:
      - looker_explore_orders
      - looker_explore_customers
    # Other props can be overridden
    owner: ...
    maturity: low
github-actions[bot] commented 1 year ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] commented 1 year ago

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

raphaelvarieras commented 1 year ago

I would like to second that request by @david-beallor and this is how I would make the case for it, in Mode Analytics in my case.

Mode uses two objects on which dashboard can be built: definitions and datasets. Ideally we would be able to identify exposures that are either one of those, as well as the dashboards that are built on top of them. The rationale is that:

Properties like name, url, tags, owner, etc would be likely different: the dataset might be owned by an analytics engineer on the team and they'd be on the hook to update it, while the dashboard may be owned by a data analyst who would want to ensure it still runs and conforms to their expectations.

tylercwood91 commented 3 months ago

I'd like to second this request as well, suggesting another use case: PowerBI Semantic Layers -> PowerBI Reports

simon-wright1 commented 3 months ago

I agree, this would be useful. We have a single Power BI semantic model feeding multiple reports, which we would like to reflect in our dbt lineage.

gradyela commented 3 months ago

I am interested in this feature / concept. While I get the termination node sentiment, many BI tools leverage a couple of concepts to get to the final dashboard. Tableau, for example, uses data sources (separate and embedded) to power views and dashboards. Somebody already mentioned looker's functionality.

It would make sense for exposures to be able to parent other exposures in many instances. It would also allow for better end-to-end documentation of the data ecosystem in a given organization.

Paul-Lees commented 2 months ago

Adding another vote, another use case in powerbi Data Flow -> PowerBI Semantic Layers -> PowerBI Reports

a dataflow can feed into many semantic model layers semantic model can feed into many reports