tblproperties and Comment are not updated for incremental

[elca-anh]

Describe the bug

My use case is a model with materialization 'incremental'. Model already exists. I am adding tblproperties option to be able to set delta.dataSkippiongStatsColumn. This property is not set. However, if I run a full refresh, the property is set. Since I checked similar behavior for the Comment that is not updated in Unity Catalog.

Steps To Reproduce

• Set or update tblproperties/comment on an existing incremental model
• dbt run
• Check in Databricks (DESC DETAIL table) or in the Databricks UI (for the comment) that the property is not set/updated

Expected behavior

Detect creation or update of the tblproerties and apply those.

Looking at dbt-databricks\dbt\adapters\databricks\relation_configs\incremental.py, config_components only contains TagsProcessor,

But:

• the materialized_view's config_components contains [PartitionedByProcessor, CommentProcessor, TblPropertiesProcessor, RefreshProcessor, QueryProcessor],
• the streaming_table's config_components contains [PartitionedByProcessor, CommentProcessor, TblPropertiesProcessor, RefreshProcessor].

But I am not sure how those are applied. For example I see that liquid clustering is applied specifically by dbt-databricks\dbt\include\databricks\macros\relations\liquid_clustering.sql I also see that tblproperties are only updated for Python (apply_tblproperties_python) in dbt-databricks\dbt\include\databricks\macros\materializations\incremental\incremental.sql

System information

The output of dbt --version:

Core: 1.8.4
Databricks: 1.8.4
Spark: 1.8.0

The operating system you're using:

Windows

The output of python --version:

Python 3.10

Additional context

Add any other context about the problem here.

[benc-db]

I thought I had already addressed this for tblproperties, but apparently not. I believe there is currently a PR for fixing it for table comments (should already work for column comments).

[benc-db]

Thanks for reporting.