littleK0i / SnowDDL

Declarative database change management tool for Snowflake
Apache License 2.0
104 stars 27 forks source link

Dynamic Tables are processed as a part of the Table Resolver #80

Closed jintaogao closed 8 months ago

jintaogao commented 8 months ago

Describe the bug Dynamic tables are getting dropped when planning/applying because table resolver is also processing the dynamic tables as tables. And the table resolver is not seeing blueprints for the dynamic table.

To recreate the bug: be in non-sandbox mode, create a dynamic table yaml, name: TEST_DT apply then run plan you'll see SnowDDL is trying to drop TEST_DT

Expected behavior the table resolver shouldn't process dynamic tables

Attach log

INFO - Resolved TABLE [TEST_DT]: DROP 
DEBUG - Resolved DYNAMIC_TABLE [TEST_DT]: NOCHANGE

Attach YAML config (if applicable) N/A

the fix should just be adding the following to here

            if r["is_dynamic"] == "Y":
                continue 
littleK0i commented 8 months ago

Could you share the top part of SnowDDL output, with cloud region and Snowflake version?

I've checked it on my sandbox accounts in AWS and version 8.9.2, and there is no flag is_dynamic in SHOW TABLES. Also, SHOW TABLES does not return dynamic tables, only SHOW DYNAMIC TABLES does.

Maybe it is something new which was not applied to all accounts yet.

jintaogao commented 8 months ago

Hi @littleK0i Snowflake version = 8.9.2 (BUSINESS_CRITICAL) Region = AWS_US_EAST_1

I also just checked, and it seemed like that SHOW TABLES no longer shows is_dynamic. I was showing as of yesterday when the issue was posted. I'll close this and reopen if it comes back

littleK0i commented 8 months ago

Yeah, this is a fundamental issue with Snowflake. Changes can be deployed and rolled back suddenly and without prior notice. It does not happen very often, but not impossible.

You may also run this command in future:

SELECT SYSTEM$SHOW_ACTIVE_BEHAVIOR_CHANGE_BUNDLES();

It may help to identify unusual active bundles which are different from "default".

For example, this is what I have now:

[{"name":"2024_01","isDefault":true,"isEnabled":true},{"name":"2024_02","isDefault":false,"isEnabled":false}]

If you see something which is "enabled", but not "default", this might be the reason for new sudden issues.

littleK0i commented 8 months ago

@jintaogao , btw, if you're planning to use dynamic tables, you may consider updating to the latest version 0.23.2. I've added some additional privileges specifically for dynamic tables.

Originally Snowflaked used SELECT future grant on tables for dynamic tables as well, but they changed it a few weeks ago.

jintaogao commented 8 months ago

@littleK0i got it, thanks for the response! https://community.snowflake.com/s/question/0D5VI000008tgIC0AY/my-dynamic-tables-are-now-creating-a-regularlooking-table-in-addition-to-the-dynamic-tableis-this-a-bug-below-is-the-code-i-used-to-create-the-dt-if-you-drop-the-one-highlighted-in-red-the-other-green-dt-also-drops this is also a related forum post i found regarding the bug