[ISSUE] Can't delete jobs+permissions resources for jobs that were created by deleted users

[alexlance]

Configuration

There are two resources of interest, a databricks_job and a databricks_permissions for that job. The two resources can get created fine, but when trying to delete both resources there is an error:

Error: cannot delete permissions: Principal name not defined

This error does not happen with all jobs there are older jobs where the error occurs, but with newer jobs, the error does not occur.

When I view the JSON of the older and newer job definitions in the Databricks web interface, I can see that the newer jobs contain a field named: creator_user_name, which the older ones do not contain.

Expected Behavior

When instructed to delete a job and its permission (by removing HCL config), these resources should be deleted.

Actual Behavior

An error occurs:

Error: cannot delete permissions: Principal name not defined

and neither the job nor the job permission get deleted.

Steps to Reproduce

Find a job that was created circa 2021, and view its JSON definition in the Databricks Web UI. If it doesn't contain a creator_user_name field, then terraform import that job and its permission into terraform. Then attempt to delete those resources by running terraform apply (the lack of hcl configuration, will result in an attempt by terraform to destroy those resources).

The job and the permission will say they are going to be destroyed, but when you type "yes" to continue, the permission refuses to destroy, and terraform halts, without destroying the job or the job permission.

Terraform and provider versions

terraform version
Terraform v1.4.6
on linux_amd64
+ provider registry.terraform.io/databricks/databricks v1.17.0
+ provider registry.terraform.io/hashicorp/aws v4.12.1

[alexlance]

Btw I've attempted to add the missing "creator_user_name" field to older jobs using both the /api/2.1/jobs/update and the /api/2.1/jobs/reset endpoints, but can't seem to set the field (which does make sense).

[alexlance]

More info - the jobs were created by someone, who then left the organization, so their Databricks account was deleted.

The job shows that no-one owns it.

That's the circumstance in which we can't delete the Job + Permissions through terraform/databricks provider.

[alexlance]

If we temporarily re-create the missing person's user account - with the same email address/username as they originally had, then the problem with deleting jobs+permissions vanishes.