EDIT
Ok, as I mixed around with this a bit more I realized that if you do table_with_connector you occasionally might want to update the indexes on the table without having to re-create the entire table by running --full-refresh.
Ergo, I refactored the on_configuration_change for the MWs to also be applied for the table_with_connector.
A similar issue was raised to dbt-core recently: https://github.com/dbt-labs/dbt-core/issues/9510
This works as expected:
apply -> drop the indexes and re-create them, without truncating the table
continue -> a warning is raised but skipped
fail -> compile error
I have verified this on our test cluster but worth to perhaps start to add tests . Let me know what you think @chenzl25.
Smol fix, adding index creation on table_with_connector for SINK INTO kind of relations.
EDIT Ok, as I mixed around with this a bit more I realized that if you do
table_with_connectoryou occasionally might want to update the indexes on the table without having to re-create the entire table by running--full-refresh. Ergo, I refactored theon_configuration_changefor the MWs to also be applied for thetable_with_connector. A similar issue was raised to dbt-core recently: https://github.com/dbt-labs/dbt-core/issues/9510This works as expected:
apply-> drop the indexes and re-create them, without truncating the tablecontinue-> a warning is raised but skippedfail-> compile errorI have verified this on our test cluster but worth to perhaps start to add tests . Let me know what you think @chenzl25.
Smol fix, adding index creation on
table_with_connectorforSINK INTOkind of relations.