search

LewisDavies / upstream-prod

A dbt package for easily using production data in a development environment.
32 stars 4 forks source link

Package not working with metrics #24

Closed TahaBelKhayate closed 7 months ago

TahaBelKhayate commented 10 months ago

[Using dbt 1.5, upstream-prod 0.7.1, dbt-labs/metrics 1.5.0]

Hi there! We've been using your package for over a year now, we are really happy with it 😁... but we are facing an issue. The overriding of ref function does not seem to work when using the dbt-labs/metrics package, especially with the metrics.calculate macro.

The macro generates a cte with a name like model_5136b70a98f4d9c12207d67556e6ef01__aggregate in the compiled code, and in this cte the table referenced in always from dev environment and never from production.

Did you have this issue raised by anyone else? Would you know a trick to correct it? I'm trying to reverse engineer the dbt-labs/metrics code but I struggle understanding what is happening.

Thanks for your help 😄

LewisDavies commented 7 months ago

Thanks @TahaBelKhayate - sorry for the big delay in responding but I'm glad to hear the package is useful!

Unfortunately I won't be able to add support for the metrics package. From a brief look at the code, it doesn't appear to use the ref function so I don't think upstream-prod would be able to redirect queries. It may require a lot of effort to unpick how it works in greater detail but I don't have the time to work on this, especially as it is no longer officially supported by dbt.

However, let me know if you'd like to investigate this yourself and I'll do my best to support.

TahaBelKhayate commented 7 months ago

Thanks @LewisDavies for your answer! Turns out we now use the manifest.json we upload to GCS on PR getting merged and download it for dbt local runs. And we are also getting away from dbt metrics anyway.

Your package served us for 2 full years, thanks for everything. Have a great week! ☀️