Any clue where / why this error might happen - "Parser Error: syntax error at or near "-" "

[DanyC97]

Hi there,

Given the latest version of duckdb and dbt-core and using the below configuration, i'm bumping into an issue where the debug error message doesn't help at all where run dbt --debug docs generate

Note, if i have different targets of type bigquery and run dbt run it compiles and works as expected, no issues. Is just when using with duckdb ....

profile.yml

# This dbt profiles.yml config file.
dwh:
    docs: # dummy duckdb adapter. Currently this is a workaround to create documentation without connecting to a real DB (Ref:https://github.com/dbt-labs/dbt-core/issues/3947)
      path: /tmp/dbt.duckdb
      type: duckdb

in dbt_profile.yml i have defined a variable as below

vars:
   gcp_billing_datasets: ['project-bq-export.gcp_billing']

in the model sql file i have

{{ config(
        materialized='incremental',
        incremental_strategy='insert_overwrite',
        partition_by={
            "field": "usage_date",
            "data_type": "DATE",
            "granularity": "DAY"
        }
    )
}}

WITH billing_data AS (
    {%- for dataset in var("gcp_billing_datasets") -%}
        {% set dataset_query %}
            SELECT CONCAT("{{ dataset }}.", table_name) AS table_name
            FROM {{ dataset }}.INFORMATION_SCHEMA.TABLES
            WHERE table_name LIKE '%resource_v1%'
        {% endset %}

and the error is

09:45:46  DuckDB adapter: Error running SQL:

            SELECT CONCAT("project-bq-export.gcp_billing.", table_name) AS table_name
            FROM project-bq-export.gcp_billing.INFORMATION_SCHEMA.TABLES
            WHERE table_name LIKE '%resource_v1%'

09:45:46  DuckDB adapter: Rolling back transaction.
09:45:46  On model.dwh.dwh_gcp_billing: Close
09:45:46  Runtime Error in model dwh_gcp_billing (models/gcp/warehouse/dwh_gcp_billing.sql)
  Parser Error: syntax error at or near "-"

Now based on my understanding i don't need to provide the backticks to FROM{{ dataset }}.INFORMATION_SCHEMA.TABLES as i don't have any special chars. However i've tried to dig into your code as well as dbt-core ( i.e https://github.com/dbt-labs/dbt-core/tree/main/core/dbt/parser ) but couldn't find much.

Due to lack of more debug info i'm a bit in the dark, could the issue arises due to the way dbt handles string interpolation and how DuckDB interprets identifiers. ?

[DanyC97]

👋 @jwills , @guenp sorry to tag you directly. Any chance you might be familiar with the codebase, if so are you able to shed some light where i can start digging further?

thank you.

[jwills]

Mmm I think there are a few problems here.

1) You definitely need to add " around the {{ dataset }} variable in the model SQL; dbt doesn't do that for you for variables, only for relations it controls. 2) I don't understand why the resulting "project-bq-export.gcp_billing".INFORMATION_SCHEMA.TABLES relation exists-- what is project-bq-export.gcp_billing? Is it a DuckDB database? Something else?