search

aws-samples / dbt-glue

This repository contains the dbt-glue adapter
Apache License 2.0
101 stars 69 forks source link

Integration test failed #469

Closed moomindani closed 3 weeks ago

moomindani commented 3 weeks ago

Describe the bug

Integration test failed: https://github.com/aws-samples/dbt-glue/actions/runs/11550774951/job/32148096036

Screenshots and log output

tests/functional/adapter/test_iceberg.py::TestIncrementalGlue::test_incremental 
=== Test project_root: /tmp/pytest-of-runner/pytest-0/project18

Invoking dbt with ['seed']
09:57:40  Running with dbt=1.8.8
09:57:40  Registered adapter: glue=1.8.6
09:57:40  Unable to do partial parsing because saved manifest not found. Starting full parse.
09:57:41  Found 1 model, 2 seeds, 1 source, 507 macros
09:57:41  
09:57:41  Concurrency: 1 threads (target='default')
09:57:41  
09:57:41  1 of 2 START seed file dbt_functional_test_9746.added .......................... [RUN]
09:57:46  1 of 2 OK loaded seed file dbt_functional_test_9746.added ...................... [CREATE 20 in 5.01s]
09:57:46  2 of 2 START seed file dbt_functional_test_9746.base ........................... [RUN]
09:57:51  2 of 2 OK loaded seed file dbt_functional_test_9746.base ....................... [CREATE 10 in 5.00s]
09:57:51  
09:57:51  Finished running 2 seeds in 0 hours 0 minutes and 10.43 seconds (10.43s).
09:57:51  
09:57:51  Completed successfully
09:57:51  
09:57:51  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2

Invoking dbt with ['run', '--vars', 'seed_name: base']
09:58:02  Running with dbt=1.8.8
09:58:02  Registered adapter: glue=1.8.6
09:58:02  Unable to do partial parsing because config vars, config profile, or config target have changed
09:58:03  Found 1 model, 2 seeds, 1 source, 507 macros
09:58:03  
09:58:03  Concurrency: 1 threads (target='default')
09:58:03  
09:58:03  1 of 1 START sql incremental model dbt_functional_test_9746.incremental ........ [RUN]
09:58:03  Glue adapter: Parameter validation failed:
Invalid type for parameter Name, value: , type: <class 'jinja2.runtime.Undefined'>, valid types: <class 'str'>
09:58:06  Glue adapter: Glue returned `error` for statement None for code SqlWrapper2.execute('''/* {"app": "dbt", "dbt_version": "1.8.8", "profile_name": "test", "target_name": "default", "node_id": "model.incremental.incremental"} */

        create or replace table glue_catalog.dbt_functional_test_9746.incremental

    using iceberg

    LOCATION '***/11550774951-2/3.10/dbt_functional_test_9746/incremental'

    as

            select * from dbt_functional_test_9746.base

  ''', use_arrow=True, location='***/11550774951-2/3.10'), Py4JJavaError: An error occurred while calling o79.sql.
: java.lang.ClassNotFoundException: 
Failed to find data source: iceberg. Please find packages at
https://spark.apache.org/third-party-projects.html
moomindani commented 3 weeks ago

I will look into this soon

moomindani commented 3 weeks ago

Integ test succeeded: https://github.com/aws-samples/dbt-glue/actions/runs/11550774951/job/32149703787