dbt-msft / dbt-synapse-serverless

dbt adapter for dbt serverless pools
MIT License
13 stars 6 forks source link

dbt-synapse-serverless

custom dbt adapter for Azure Synapse serverless pools. This adapter largely inherits from dbt-synapse, which itself inherits from dbt-sqlserver. For more info, see those repos.

major differences b/w dbt-synapse-serverless and dbt-synapse

In serverless pools, you can't:

This is not published to PyPI. for now, install from Github with:

pip install git+https://github.com/dbt-msft/dbt-synapse-serverless.git

Caveats

  1. You can't use the default or master database on a "built-in" serverless pool, because somehow they enmeshed with the spark pool. You must go to the master db and make a new db first. That is what you will use for the dbt project.
  2. dbt won't stop you from trying to make tables, but it's not going to work. I would welcome PRs if people wanna make that experience better
  3. I don't expect this to be supported for much longer as changes to dbt-core will require tables to make things like tests work.

Authentication

Please see the Authentication section of dbt-sqlserver's README.md.

The only difference is to provide the adapter type as synapseserverless so for example:

jaffle_shop:
  target: serverless
  outputs:
    serverless:
      type: synapseserverless
      driver: "ODBC Driver 17 for SQL Server"
      schema: dbo
      host: <serverlessendpoint>
      database: <serverlessdb>
      authentication: CLI