search

dbt-labs / dbt-snowflake

dbt-snowflake contains all of the code enabling dbt to work with Snowflake
https://getdbt.com
Apache License 2.0
297 stars 177 forks source link

[Bug] Seed file already exists when I run dbt-build with full refresh model #1071

Closed Evelyn-yx-Li closed 5 months ago

Evelyn-yx-Li commented 5 months ago

Is this a new bug in dbt-snowflake?

Current Behavior

When I was running dbt build --target snowflake-ci --full-refresh using the Github Action, it gave out an error like 

Database Error in seed mock_1 (seeds/mock_1.csv)
  002002 (42710): SQL compilation error:
  Object "mock_1"' already exists.

It never happens when I was working on dbt v1.6

Expected Behavior

I'm assuming the seed files will not exist when I run with full-refresh mode? And it doesn't happen to any other adapters for me so far And I also checked it would no error when I downgrade Snowflake to v1.7

Steps To Reproduce

  1. Run dbt clean&&dbt deps
  2. Run dbt build --full-refresh --target snowflake-ci
  3. It said seed files already exists

Relevant log output

No response

Environment

- OS: Mac
- Python: 3.11
- dbt-core: 1.8.1
- dbt-snowflake: 1.8.2

Additional Context

No response

jtcohen6 commented 5 months ago

Thanks for the report @Evelyn-yx-Li.

We recently made a change that is potentially relevant, using show objects instead of show terse objects to populate the cache:

However, since your seed is being created as a conventional table in Snowflake, I don't see why the result of show objects would be different from show terse objects in terms of the metadata reporting on the existence of that table.

Is this something you're able to reliably reproduce? If so, would you be able to provide a simple reproduction case? That would help us significantly to identify the root cause.

Evelyn-yx-Li commented 5 months ago

Hi @jtcohen6, thanks for your quick reply! Actually I didn't make any code change. The error came out after I pip install -r requirements.txt in which 

dbt-core==1.8.1
dbt-redshift==1.8.0
dbt-bigquery==1.8.1
dbt-snowflake==1.8.2

and then dbt seed --target snowflake-ci --full-refresh

dbt seed --target snowflake-ci --full-refresh                                                                   1 ↵
23:07:26  Running with dbt=1.8.1
23:07:26  Registered adapter: snowflake=1.8.1
23:07:26  Found 11 models, 28 data tests, 2 seeds, 2 sources, 649 macros
23:07:26  
23:07:28  Concurrency: 2 threads (target='snowflake-ci')
23:07:28  
23:07:28  1 of 2 START seed file mock_1  [RUN]
23:07:28  2 of 2 START seed file mock_2  [RUN]
23:07:29  1 of 2 ERROR loading seed file mock_1  [ERROR in 0.78s]
23:07:29  2 of 2 ERROR loading seed file mock_2  [ERROR in 0.89s]
23:07:29  
23:07:29  Finished running 2 seeds in 0 hours 0 minutes and 2.83 seconds (2.83s).
23:07:29  
23:07:29  Completed with 2 errors and 0 warnings:
23:07:29  
23:07:29    Database Error in seed mock_1 (seeds/mock_1.csv)
  002002 (42710): SQL compilation error:
  Object 'xxxx."xxxx"."mock_1"' already exists.
23:07:29  
23:07:29    Database Error in seed mock_2 (seeds/mock_2.csv)
  002002 (42710): SQL compilation error:
  Object 'xxxx."xxx"."mock_2"' already exists.

The seed file is just a very simple csv file with about 10 columns. It's used for our CI workflow within Github Action. It happened both locally and in CI. I think a way to produce is to have a csv file and run the command. And the issue doesn't happen to dbt-bigquery v1.8, as well as dbt-snowflake v1.7 (v1.7.8) or 1.6. Thanks!

Evelyn-yx-Li commented 5 months ago

Hi @jtcohen6, just checked, it runs well with dbt-snowflake==1.8.3, thus it's good to close the issue. Thank you!

amychen1776 commented 5 months ago

Thanks for following up! Closing now