search

databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
456 stars 393 forks source link

[Fix] Remove single-node validation from interactive clusters #4222

Closed shreyas-goenka closed 1 week ago

shreyas-goenka commented 1 week ago

Changes

There were reports on https://github.com/databricks/cli/issues/1546 about customers being unable to use cluster policies for single-node job clusters because Terraform was performing this validation. https://github.com/databricks/terraform-provider-databricks/pull/4216 removed this validation for job clusters. This PR removes this validation for interactive clusters as well to keep them consistent and unblock the use of policy from interactive clusters.

Also, the clusters team is improving the API interface to simplify creating single-node clusters, significantly reducing the chances of users misconfiguring them.

Tests

Unit test and manually. Clusters are now successfully created and updated with num_workers=0

github-actions[bot] commented 1 week ago

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger: go/deco-tests-run/terraform

Inputs:

Checks will be approved automatically on success.

eng-dev-ecosystem-bot commented 1 week ago

Test Details: go/deco-tests/11818584690

alexott commented 1 week ago

@tanmay-db what is your opinion giving that you're porting clusters to the plugin framework?

pietern commented 1 week ago

@shreyas-goenka The PR you reference talks about job clusters. Am I missing the ref to interactive clusters?

shreyas-goenka commented 1 week ago

@pietern, you are right. The issue only discusses job clusters. There's no reference for interactive clusters, but now that we support interactive clusters via DABs, we should solve this issue here as well and keep them consistent.

Updated the PR description to say as much.

tanmay-db commented 1 week ago

@tanmay-db what is your opinion giving that you're porting clusters to the plugin framework?

Hi @alexott, it looks good, I will update the plugin framework implementation to not have this validation.

shreyas-goenka commented 1 week ago

@tanmay-db We'll be adding this validation to DABs as a warning instead of a warning error. Maybe that's something we should do in the plugin framework as well if supported.