elementary-data / elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
https://www.elementary-data.com/
Apache License 2.0
1.94k stars 165 forks source link

[ELE-422] Exclude packages models from dashboard #695

Open Maayan-s opened 1 year ago

Maayan-s commented 1 year ago

Is your feature request related to a problem? Please describe. Today the dashboard includes data about the Elementary (and other packages models). Elementary are not controlled by the project owner, so no reason to show in the report. However, some packages are part of the owner's dbt project (practically, not technically) so other packages should probably be subject to configuration with exclude as default.

image

Additional context Slack thread - https://elementary-community.slack.com/archives/C02CTC89LAX/p1676892610316469 (Willi Mueller, leila)

ELE-422

elongl commented 1 year ago
  1. How common is it that dbt users define other projects as packages?
  2. When doing so, do they not generate report for the other projects? Because otherwise we could claim that they'll see it on the other reports.
Maayan-s commented 1 year ago

Updating that from 0.7.3 the Elementary models are excluded.

About your questions @elongl -

  1. We saw several users, mostly in big envs, managing their dbt project in packages. It's a way to manage "different repos" without the downsides of different projects. it's their way to overcome the fact dbt doesn't support multi project yet.
  2. No. They use it as a single project.
willi-mueller commented 11 months ago

Hi, I'm just introducing elementary in a new project for a new customer 💫 Just like 10 months ago I stumbled upon this issue. I've got 9 models and 8 of them have tests, but the dashboard shows that 16 don't have any tests. What could be the reason?

From my point of view, just excluding all external packages would be great and would prevent new users from doubting the edr report.

After running edr report for the first time, I am surprised that so many tables are not tested. However, the tables not tested come from packages other than mine.

My packages:

packages:
  - package: dbt-labs/dbt_utils
    version: 1.1.1
  - package: calogica/dbt_expectations
    version: 0.10.1
  - package: elementary-data/elementary
    version: 0.13.0

Screenshot 2023-12-11 at 18 38 35