z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 45 forks source link

How do you run dbt-power-user with dbt-osmosis? #29

Closed evanaze closed 1 year ago

evanaze commented 1 year ago

In this issue @z3z1ma you say that the preferred method of development is with the dbt-server, and in the server docs you imply that you can use the server with dbt-power-user. I've tried to stitch together the solution but am coming up short.

Maybe you could provide an explanation for how to make the integration work and update the docs? Right now dbt-power-user points to this repo for how to set up a dbt REPL, so if you could make that part of the docs more clear I think it could help other people as well.

mdesmet commented 1 year ago

dbt power user doesn't depend anymore on dbt-osmosis. We removed the dependency because it required the user to install additional packages, so we had to accommodate for both cases (not having dbt-osmosis installed and having it installed). As the extension is used by a lot of users (around 30K) in very different environments (for example locked down corporate environments) with any supported dbt adapter. It is too cumbersome to require installation of additional packages or troubleshoot versioning issues.

That's why now dbt power user just uses a simple python file, included with the extension, that provides the dbt osmosis features needed for dbt power user and loads it in a long-running python process using only Python itself, dbt-core and its adapters). The long-running process has the benefits of the server: fast execution due to heavily caching the manifest, without having to install any additional Python dependencies.

This ensures that every dbt power user simply gets the same solid experience through the standard visual code marketplace updates instead of being prompted to update or install additional packages and the version management that comes with it.

A user wanting to use dbt-osmosis can still install it, it won't impact dbt power user, that way you can also enjoy the great features that dbt-osmosis provides.

z3z1ma commented 1 year ago

Closing this as a non-issue. I will work with Michiel to facilitate updating vendored code as the guts of dbt-osmosis move to https://github.com/z3z1ma/dbt-core-interface