Command terminates without error

[mmmatthew]

When I run the following command, the session is established but then the process ends without any message:

dbt-metabase models \
    --dbt_manifest_path ./target/manifest.json \
    --dbt_schema dbt_prod \
    --metabase_host $url \
    --metabase_user $metabase_email \
    --metabase_password $metabase_pw \
    --metabase_database $dbname \
    --dbt_database dbt_prod

2021-09-14 09:42:07,987 - INFO - Session established successfully

Am I doing something wrong?

[z3z1ma]

Can you try adding --verbose to the command and seeing what happens? I would only imagine the command terminating if it found no objects to loop through matching your supplied dbt_prod schema/database. We can tell more with the flag though.

[mmmatthew]

Thanks for the answer! Indeed, it didn't find the models in dbt_prod. Surprisingly:

• It also tried to look for ephemeral models
• If I run the same command but with --dbt_path instead of --dbt_manifest_path, it works fine.

[z3z1ma]

Wanted to get some movement here:

It also tried to look for ephemeral models

What is the exact output? With a verbose flag, what it actually says is something like "Skipping ephemeral model IMD_MONDAY_SCHEDULING_HISTORY not manifested in database". It doesn't mean its looking for ephemeral, just being very verbose.

With the folder parser, dbt_path pulls in all files since it cannot resolve schema except for in source files and even then only if the source file is not using jinja for its schema: key. The manifest parser on the other hand is able to use schema to filter the subset of models early.

So the no-op you are experiencing almost has to come from the dbt_schema or dbt_database arguments not matching up to what's in the manifest.json Very likely that at least. We need an answer to that before we can rule this an issue.

Best way might be to control-F in a manifest artifact and inspect the schema field as well as the database field in pretty much any model

[mmmatthew]

Thank you Alexander for your assistance! Indeed, the problem appears to be with the schema names. FYI, I am using BigQuery as a data store. It seems like in the build_metadata_lookups() method falls back to using public, even though the schema is defined in dbt. I was able get the synchronization to work by fixing schema_name to 'PUBLIC' in export_model(). The resolution of foreign keys still doesn't work.

As a next step I will check what Metabase is returning to see if maybe the schema isn't being identified correctly.

[mmmatthew]

I can confirm that metabase is returning null for table schemas, but in dbt the schema is defined as dbt_prod

[z3z1ma]

Ahh I think its coming back to me @mmmatthew, Doesn't metabase api return dataset as a key in the json response if using bigquery instead of schema? Would it be possible for you to confirm the network response from hitting this endpoint?

https://your_metabase_host/api/table/{id}

where id is the id of a table

you can find table ids here pretty easily

https://your_metabase_host/reference/databases/2/tables where 2 is the id of whatever dataset has the table

If this is the case, it might be an easy fix.

[mmmatthew]

We're getting closer! I found both the project_id (corresponds to dbt_database) and dataset_id (corresponds to dbt_schema) in the metabase responses, under ./details when hitting the api/database/{id}/metadata endpoint or under ./db/details when hitting the api/table/{id} endpoint.

{
...
 "details": {
        "use-service-account": null,
        "dataset-id": "dbt_prod",
        "service-account-json": "**MetabasePass**",
        "use-jvm-timezone": false,
        "include-user-id-and-hash": false,
        "ssl": true,
        "project-id": "bigquery_project",
        "let-user-control-scheduling": true
    },
...
}
`

[mmmatthew]

I would be happy to make a PR for this, btw, once we agree on how to fix it.