Contributing

[joshuataylor]

I've also been futzing around the last few weeks with a dbt Plugin, aiming to support PyCharm/IntelliJ with similar features to what the official plugin has, but in a less... janky way.

Anyway, as you're way ahead of what I've worked on, I would like to submit some of my features/ideas, pending your feedback, and help add code.

I've been using dbt for a while, and also think that the dbt vscode plugin has some nifty ideas.

My working prototype + dummy project, where I removed some features to have it in a working state. My initial goal way to just show the model run results in the query editor :).

Database Tools Integration

One great feature Jetbrains has with their IDEs is the Database Tools plugin (aka DataGrip), it's just a shame it's locked away and their documentation is basically "figured it out".

I have always wanted "inline execution" for a model, where you can have a Run Configuration and it would compile and run that model.

Steps required:

1. Compile model
2. Take the compiled SQL (or path) and pass it into the query editor
3. Execute it

Problems I've found with this "simple approach":

1. Doing a basic dbt compile also has extra output, even with format.
2. The query console/services tool window is.. finnicky.

I ended up creating a Run Configuration that executes a Python file in the same virtualenv that the dbt runs in. Kinda gross, but it's probably the most stable way going forward. incredibly hacky gist I quickly threw together as a PoC, note I also commented some extra fields I would like to add to the tool window output at some point. (I serialise it as JSON, as it's probably the best solution going forward, parsing the terminal output is rough).

For the run configuration, I forced the use of a DataSource, and check that it's correctly configured.

I didn't end up creating a project configuration, I just stumbled across this repo.

It ends up working quite well, the Query Executor was a bit of a confusing mess to get to work (I also don't know Kotlin, but have enough experience with other languages to get going). I also ended up doing it as a LightVirtualFile, then had to check the service for the last file, which is a bit of a hack but is fine for a PoC :).

Database Specific Features

I don't think there should be a need a to make the tool database specific, but with a correctly configured dbt envirorment & Data Source, you should be able to execute whatever SQL you throw at it.

Testing Integration

I would love to display the dbt testing output the same way Java/Python does (screenshot stolen from Google Images):

--

Anyway, braindump over, let me know your thoughts.

[rinchinov]

Hi!

My first tries and approaches to execute queries were very similar to yours. But I met issues with how idea works with sql files when jinja is enabled for .sql file -- idea just ignored that it is sql and I wasn't able to run queries in console. I tried it at 2023 version, it could be great if at 2024 this behaviour was changed(I need to check it). So that why I decided to pause this activity and implement queriyng in with workarounds(with own query manager and ui for it).

But I agree that in general approach with database tools is better.

I will read your code carefully, especially part with accessing datasource and run queries in console looks promising

UPD: maybe this PR will solves problem that I had before https://github.com/rinchinov/ij-dbt-plugin/pull/52 , but I'm not sure 🙃

UPD2: I checked it, at 2024 and with #52 change. Once I adding jinja2 templating for .sql, idea disables run button(in datagrip panel at top of editor) for .sql file 😢

[rinchinov]

Hey! @joshuataylor

I read your code, it is really usefull, especially this part could be useful

I will integrate this into my plugin soon 🙏

PS: I already have the draft that works, I am going to do more checks and then publish it

[rinchinov]

Hey!

In the last last release I've integrated this mechanism into plugin.

Thank you for good idea of feature 👍