databricks / cli

Databricks CLI
Other
132 stars 50 forks source link

Add a warning when UC resource references or names contains capital letters #1763

Open shreyas-goenka opened 3 weeks ago

shreyas-goenka commented 3 weeks ago

Using a capital letter can cause a permanent drift in the state because the APIs are often case insensitive, but internally, UC resources are always lowercase.

For instance, in case of catalogs in a DLT pipeline, this can cause the pipelines to always be recreated and recomputed in DABs.

shreyas-goenka commented 3 weeks ago

Alternately, we can start normalizing all UC resource names to lowercase on the client side and avoid the warning/overhead for the users altogether.

andrewnester commented 3 weeks ago

I think we should instead just normalise it

shreyas-goenka commented 3 weeks ago

It's also the option I prefer. We need to however:

  1. Compile a list of all applicable fields. It's going to be more than catalog for DLT pipelines. Eg: target for DLT.
  2. Sanity check with the owning (or UC) team that these fields will not become case-sensitive in the future.