gouline / dbt-metabase

dbt + Metabase integration
https://pypi.org/project/dbt-metabase/
MIT License
439 stars 63 forks source link

Exposures have no dependencies. depends_on arrays are empty #260

Open ampersand-five opened 1 week ago

ampersand-five commented 1 week ago

I updated dbt-metabase from 1.0.1 to 1.3.1 and after doing that when I run exposures, all the depends_on arrays are now empty, where as before they were not. Which makes the dependency graph flat now, i.e. broken.

Any ideas as to why the depends_on in exposures is now not populating?

gouline commented 1 week ago

I need more details to reproduce this:

  1. dbt version(s)
  2. Metabase version(s)
  3. Database type(s), e.g. PostgreSQL, Snowflake, BigQuery
  4. Exposures type(s), e.g. dashboard, question from a table, question from another question, native SQL query

Can you also try other versions between 1.0.1 and 1.3.1 to narrow down when the bug was introduced?

ampersand-five commented 1 week ago

1) python = "3.10" dbt-snowflake = "1.7.1" dbt-metabase = "1.3.1" dbt-core = "1.7.6"

2) Metabase 1.49.14 3) Snowflake 4) A full exposure dump, so all of them? We have everything in our Metabase, the exposure file ends up ~15k lines.

Other versions:

gouline commented 1 week ago

Dependencies work up to dbt-metabase 1.1.0 for me. After that, the dependencies stop populating.

Are you saying 1.1.0 is the last version working or the first version not working?

I can't reproduce your problem with the sandbox project running the same versions of dbt, Metabase and Snowflake, not sure what's different about your environment.

Assuming 1.1.0 is where it started though, #228 seems like the most likely culprit. Here's my blind attempt at reverting that change in #262, can you try installing that branch and see if that fixes it?

pip install git+ssh://git@github.com/gouline/dbt-metabase.git@fix/260
ampersand-five commented 1 week ago

Oh I see, that's not clear. I edited my response. It works in 1.0.3, but in the next release of 1.1.0 it does not work.

I'll try that branch.

ampersand-five commented 1 week ago

That did not fix it.

ampersand-five commented 1 week ago

🤔 I'm having a different error on 1.0.3 now, that, you tell me, if this new error I have, is related to the missing dependencies error or not:

dbt-metabase exposures runs fine (when I'm using dbt-metabase 1.0.3) with dependencies populated, but when I run dbt docs generate I get an error where:

Exposure 'exposure.ex.__b_example__data' (models/exposures/exposures.yml) depends on a source named 'example_db_b.example_table' which was not found

This error, I think, is because: I have three databases connected in Metabase, let's say database A, B and C. I have dbt configured to only look at database A. In Metabase we have custom SQL questions written for all three. This error is failing on a question in Metabase that is written for database b, which is not in dbt's output because I only have it on database a.

Is this related or different? If it's different I'll make a new issue.

gouline commented 1 week ago

Not much I can do without being able to reproduce this, unfortunately. See if you can write a query that exhibits your 1.3.1 issue against the sandbox project or your own minimal sample project that you can share as a repository (removing anything proprietary), then I can have a look.

gouline commented 1 week ago

To help you troubleshoot, I created a debug branch exposure-depends-dump to try. This writes dependencies missing from the dbt model lookup as unknown('table') in the exposure YAML and dumps those lookups in debug.yml:

pip install git+ssh://git@github.com/gouline/dbt-metabase.git@exposure-depends-dump