Dataform tools - a vscode extension
⚠️ This is not an officially supported Google product.
VS Code extension for Dataform with following features
| Feature | Description |
|---|---|
| Compiled Query & Dry run stats | Compiled query in a vertical split |
| Dependancy graph | Interative dependancy graph with external sources higlighted in distinct colors |
Inline diagnostics on .sqlx file ❗ |
Native lsp like experience with diagnostics being directly put on both the sqlx file & compiled query |
| Preview query results | Preview query results in a table by running the file |
| Go to definition | Go to definition for source in $ref{("my_source")} and javascript blocks in .sqlx files |
| Auto-completion | - declarations in ${ref("..")} trigger when $ character is typed - Dependencies when " or ' is typed inside the config block which has dependencies keyword is in the line prefix - tags when " or ' is typed inside the config block which has tags keyword is in the line prefix |
| Code actions | Apply dry run suggestions at the speed of thought |
| Run file(s)/tag(s) | Run file(s)/tag(s), optionally with dependencies/dependents/full refresh using vscode command pallet / menu icons |
| Format using Sqlfluff 🪄 | Fromat .sqlx files using sqlfluff |
| BigQuery snippets | Code snippets for generic BigQuery functions taken from vscode-langauge-sql-bigquery extension |
Requirements
-
npm i -g @dataform/cliRun
dataform compilefrom the root of your Dataform project to ensure that you are able to use the cli -
To enable formatting using sqlfluff install sqlfluff
# install python and run pip install sqlfluff -
To enable prettier diagnostics install Error Lens extension [ optional ]
[!NOTE] Trouble installing ? Please see FAQ section, if you are still stuck, please raise an issue here
Features
Compiled query & Dry run stats
Dependency graph
Inline diagnostics errors on .sqlx files ❗
Preview query results
Go to definition
Go to definition for source in $ref{("my_source")}. Takes you to my_source.sqlx or sources.js at the line where my_source is defined. There is also support for go to definiton
from a javascript variable/module from a .sqlx file to js block or .js file where the virable or module declaration exsists
Autocomplete model, tags, dependencies
Auto completion support for dependencies when " or ' is typed inside the config block which has dependencies keyword is in the line prefix
Declarations in ${ref("..")} trigger when
Auto completion support for tags when " or ' is typed inside the config block which has tags keyword is in the line prefix
Formatting using sqlfluff
Run file/tag with dependencies/dependents
Open vscode command pallet by pressing CTLR + SHIFT + p or CMD + SHIFT + p on mac and run one of the required commands
| Commands |
|---|
Dataform: Run current file |
Dataform: Run current file with dependencies |
Dataform: Run current file with dependents |
Dataform: Run current tag |
Dataform: Run current tag with dependencies |
Dataform: Run current tag with dependents |
Dataform: Format current file |
Dataform: Run file(s) / tag(s) with options |
Known Issues
- [ ] Features such as go to definition / dependancy graph might not work with consistantly with
${ref("dataset", "table")}or when it is multiline or a different format works best with${ref('table_name')}format
TODO
- [ ] Bundle javascript files in the extension using esbuild or webpack
- [ ] Handle case where user is not connected to internet or on vpn where network request for dry run cannot be made