ij-dbt-plugin

Alt text

Key features
Quick start
Full Setup instructions
Configuration options reference
Query execution mechanism explanation
DBT for idea IDEs

Elevate Your Data Engineering Workflow

The DBT plugin for PyCharm revolutionizes the interaction with DBT projects for data engineers and analysts. This indispensable tool integrates advanced DBT navigation and documentation preview directly into your IDE.

Key Features

Synchronized Navigation: Navigate through DBT projects with enhanced capabilities. The plugin intelligently highlights corresponding entries in DBT documentation as you explore models, sources, macros and tests within PyCharm. This seamless integration streamlines workflows and enriches project comprehension.
Autocompletion: Improve efficiency with the ability to write code with ability to autocomplete jinja macros and arguments for ref and source macros.
Copy/Paste Feature: Improve efficiency with the ability to copy and paste queries, seamlessly replacing refs/sources for database operations.
Run selected query: Improve efficiency with the ability to run query parts right inside the IDE.
Integrated DBT Documentation Preview: Access DBT documentation without leaving PyCharm. A dedicated plugin window provides instant insights and information relevant to your current context, enriching your development experience with immediate access to your project's data documentation.
Annotations: Annotations for macros, models, and sources

Leverage these features to transform your navigation and documentation review processes, boosting productivity and understanding of DBT projects.

Quick start

Install the plugin
Open some dbt project
Make sure that in IDE set python interpreter with dbt
If your dbt_project.yml not in root directory of project
- set project file path in plugins settings
If your profiles not in ~/.dbt
- set path to profiles directory in plugins settings

Full Setup Instructions

Prerequisites

Ensure you have a Python SDK configured in IDE with DBT installed. This setup is essential for the correct functioning of the plugin.

Install the DBT Plugin

Install the DBT plugin directly from the PyCharm Marketplace to get started.

Configure Python SDK

Navigate: Go to File > Project Structure > Project.
Set SDK: Choose a Python interpreter where DBT is installed. If DBT is not installed, install DBT accordingly.

Specify DBT Project Paths

DBT Project Settings: Access Preferences > DBT Project Settings to set both the DBT interpreter path (your DBT project directory) and the DBT profile path (your DBT profile location).

Open and Navigate Projects

Open any DBT project in PyCharm to automatically unlock advanced navigation features, making every DBT component effortlessly accessible.

Optional: Restart PyCharm/IntelliJ IDEA

A restart may be necessary to fully integrate the new settings, optimizing your plugin experience.

Configuration options reference

Main settings

Settings name	Description	By default	Examples
dbt_project.yml File Path	Path to dbt project file(dbt_project.yml). DBT docs link	dbt_project.yml
dbt Profile Directory	Path to directory where dbt profile file is placed. DBT docs link	~/.dbt
DBT runner import	Import string, will be used to call dbt commands see programmatic invocations	from dbt.cli.main import dbtRunner	from custom_package import customDbtRunner
DBT interpreter path	Python interpreter SDK where DBT is installed, if empty it takes current project's sdk
DBT environment variables	Environment variables that will be used for dbt calls		DB_PASS=secret1@;DB_USER=dbt
Collect usage statistic	Statistics help to analyze which features most used, which adapters and environments are most popular to prioritize plugin development. privacy policy	unchecked

Query execution settings

String templates are used for query formatting during execution, %s -- selected text. See section with query execution description

Settings name	Description	By default
Query paginated template	for pagination to get paginated result	`%s LIMIT ? OFFSET ?`
Query count template	for pagination to get count of pages	`SELECT COUNT(*) FROM (%s)`
Query plan template	for getting query plan	`EXPLAIN %s`
Query dry run template	for dry run query, if adapter supports	`%s LIMIT 10`

Query execution mechanism

Probably query execution is the most difficult and not obvious part of the plugin. It is adapter specific, so it can be different in different adapters.

Now there is two ways how to execute query:

With IDEA's database tools
With plugins owns query execution logic
- Run query(with pagination)
- Dry run query
- Get query plan

How to run query for selected text in editor:

Alt text

If your adapter is postgres then set up datasource to with name {project name}__{target}, e.x. jaffle_shop__dev, plugin will use it.
Select query in editor and one of:
- Right click and select Run Selected Query
- Or click on top in run configurations
Choose one of:
- Run with database tools: {your desired target} to run query in IDEA's database tools
- Run: {your desired target} to run query with pagination
- Query plan: {your desired target} to get query plan
- Dry run: {your desired target} to dry run query
Plugin should replace refs and sources and show query execution results in plugins tool window

Note: It formats queries with query templates from plugin's settings.

If your adapter is postgres then plugin takes JDBC connection from IDEA's database tools, and runs query with JDBC.

For the rest adapters it uses dbt some tricky logic to run query with dbt cli(it works much slower than JDBC).

Plugin based on the IntelliJ Platform Plugin Template.

rinchinov / ij-dbt-plugin

readme