duckdb / dbt-duckdb

dbt (http://getdbt.com) adapter for DuckDB (http://duckdb.org)
Apache License 2.0
790 stars 70 forks source link

Runtime Error: `duckdb.duckdb_extension` Extension missing or invalid #331

Closed jagrafft closed 4 months ago

jagrafft commented 4 months ago

dbt run throws a runtime error saying the extension duckdb.duckdb_extension cannot be loaded from ${env:HOMEPATH}\.duckdb\extensions\v0.9.2\windows_amd64\. Happens in all cases, and https://duckdb.org/docs/extensions/overview does not list an md extension.

System

Error Message

22:43:57  Running with dbt=1.7.7
22:43:57  Checking ${env:HOMEPATH}\dev\python\duckdb-dbt-demo\dbt_packages/*
22:43:57  Cleaned ${env:HOMEPATH}\dev\python\duckdb-dbt-demo\dbt_packages/*
22:43:57  Checking ${env:HOMEPATH}\dev\python\duckdb-dbt-demo\target/*
22:43:57  Cleaned ${env:HOMEPATH}\dev\python\duckdb-dbt-demo\target/*
22:43:57  Finished cleaning all paths.
22:44:01  Running with dbt=1.7.7
22:44:02  Registered adapter: duckdb=1.7.1
22:44:02  Unable to do partial parsing because saved manifest not found. Starting full parse.
22:44:03  Found 1 model, 2 sources, 0 exposures, 0 metrics, 391 macros, 0 groups, 0 semantic models
22:44:03
22:44:07
22:44:07  Finished running  in 0 hours 0 minutes and 4.87 seconds (4.87s).
22:44:07  Encountered an error:
Runtime Error
  IO Error: Extension "${env:HOMEPATH}\.duckdb\extensions\v0.9.2\windows_amd64\duckdb.duckdb_extension" not found.

  Candidate extensions: "md"

profiles.yml

analytics:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: '/path/to/folder/with/data/dbs/analytics.duckdb'
      attach:
        - path: '/path/to/folder/with/datadbs/source.duckdb'
          type: duckdb
jwills commented 4 months ago

Pretty sure md refers to the motherduck extension which ships by default; I'm really confused by this one. Does it go away if you exclude the attach configuration block from your profile? Or if you keep the attach block but remove the type: duckdb option?

jagrafft commented 4 months ago

@jwills Removing the type key from the attach object seems to work. Thank you!!

Are you able to elaborate on why this might work?

Updated profiles.yml

analytics:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: '/path/to/folder/with/data/dbs/analytics.duckdb'
      attach:
        - path: '/path/to/folder/with/datadbs/source.duckdb'
jwills commented 4 months ago

@jagrafft tbh no, it was just a guess based on what you were doing and the error that you were seeing. The dbt-duckdb code is dumb about the types of attached databases b/c people keep adding new ones (e.g., mysql was recently added), so we just treat it as a string.

jagrafft commented 4 months ago

@jwills The strategy makes sense if dbt-duckdb isn't yet stable. I'll bear your advice in mind if I run into other bugs. Thanks!