Open luukschagen opened 4 months ago
I am facing the same issue. Following.
Turns out the issue above is caused by the configuration using spaces instead of underscores in the privileges. Indeed the tf provider documentation correctly states that it should be with underscores, but the 'official' names of the privileges from the general Databricks documentation uses spaces.
I guess the core of the issue is still in the fact that the version with spaces does technically 'work' in that it applies the grants , but just the diff check fails.
I suppose this could be fixed either by properly allowing the version without underscores as the pull request https://github.com/databricks/terraform-provider-databricks/pull/3292 aims to do. Or just by erroring properly, with a bit more explicit error message when applying a configuration with spaces.
So I'll leave the issue open for now, because the inconsistent behavior currently should probably still be considered a bug
@luukschagen Thanks for raising. I'm reaching out to our internal team to see if they can update the documentation and error message to make them consistent.
I am facing the same issue and it is quite annoying :) Adding the underscores makes the magic but still it will be great if this is fixed.
USE CATALOG
should be replaced with USE_CATALOG
This should resolve the issue.
When provisioning databricks_grant resources (and I believe the same thing occurs on databricks_grants (plural) resources, though this is less extensively tested), the resources seem to get created fine on Databricks, but the provider appears to keep looping and waiting for the update, until it reports with an error that the grants don't match. Subsequent runs then error with a different error, saying that the grants are duplicate.
Configuration
Expected Behavior
After running the above minimal snippet, I expect to see the metastore connected to the workspace, with one created catalog called 'testcatalog', and the group 'account users' having ALL PRIVILEGES on this catalog, with terraform exiting successfully.
Actual Behavior
In reality, all of the abovementioned happens correctly, except that terraform errors out with the following error message.
The actual resources on databricks are created correctly. But it appears that the 'diff' check between the existing and desired resources erroneously reports that the states don't match, even though they do: https://github.com/databricks/terraform-provider-databricks/blob/a53018ed83b7aec1a27703c8c40be8b222b346ba/catalog/permissions/permissions.go#L63
Subsequent runs therefore fail again, but now with a different error, reporting that the grants already exist
Steps to Reproduce
Terraform and provider versions
Terraform v1.7.4 on windows_amd64
databricks/databricks/1.37.1/windows_amd64
Is it a regression?
Don't know
Debug Output
Debug Logs
``` 2024-02-29T14:06:12.080+0100 [INFO] Starting apply for databricks_grant.test_all 2024-02-29T14:06:12.081+0100 [DEBUG] databricks_grant.test_all: applying the planned Create change 2024-02-29T14:06:12.380+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: GET /api/2.1/unity-catalog/permissions/catalog/testcatalog < HTTP/2.0 200 OK < {}: @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=3af255ed-7cea-2e78-4b9e-9886ab3673e8 tf_resource_type=databricks_grant @module=databricks tf_rpc=ApplyResourceChange timestamp="2024-02-29T14:06:12.379+0100" 2024-02-29T14:06:12.695+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: PATCH /api/2.1/unity-catalog/permissions/catalog/testcatalog > { > "changes": [ > { > "add": [ > "ALL PRIVILEGES" > ], > "principal": "account users" > } > ] > } < HTTP/2.0 200 OK < { < "privilege_assignments": [ < { < "principal": "account users", < "privileges": [ < "ALL_PRIVILEGES" < ] < } < ] < }: tf_req_id=3af255ed-7cea-2e78-4b9e-9886ab3673e8 @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 @module=databricks tf_provider_addr=registry.terraform.io/databricks/databricks tf_resource_type=databricks_grant tf_rpc=ApplyResourceChange timestamp="2024-02-29T14:06:12.695+0100" 2024-02-29T14:06:12.695+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: Waiting for state to become: [success]: timestamp="2024-02-29T14:06:12.695+0100" 2024-02-29T14:06:12.995+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: GET /api/2.1/unity-catalog/permissions/catalog/testcatalog < HTTP/2.0 200 OK < { < "privilege_assignments": [ < { < "principal": "account users", < "privileges": [ < "ALL_PRIVILEGES" < ] < } < ] < }: tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=3af255ed-7cea-2e78-4b9e-9886ab3673e8 tf_resource_type=databricks_grant tf_rpc=ApplyResourceChange @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 @module=databricks timestamp="2024-02-29T14:06:12.994+0100" 2024-02-29T14:06:12.995+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: Permissions for catalog-testcatalog are: &{[{account users [ALL_PRIVILEGES] [Principal]}]}: timestamp="2024-02-29T14:06:12.994+0100" 2024-02-29T14:06:13.813+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: GET /api/2.1/unity-catalog/permissions/catalog/testcatalog < HTTP/2.0 200 OK < { < "privilege_assignments": [ < { < "principal": "account users", < "privileges": [ < "ALL_PRIVILEGES" < ] < } < ] < }: tf_provider_addr=registry.terraform.io/databricks/databricks tf_rpc=ApplyResourceChange @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 @module=databricks tf_resource_type=databricks_grant tf_req_id=3af255ed-7cea-2e78-4b9e-9886ab3673e8 timestamp="2024-02-29T14:06:13.813+0100" 2024-02-29T14:06:13.813+0100 [DEBUG] provider.terraform-provider-databricks_v1.37.1.exe: Permissions for catalog-testcatalog are: &{[{account users [ALL_PRIVILEGES] [Principal]}]}: timestamp="2024-02-29T14:06:13.813+0100"Important Factoids
Would you like to implement a fix?
Not proficient enough in go to contribute to a fix.