azurerm_data_factory_linked_service_azure_databricks single node

[datacom-bozhu]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.0

AzureRM Provider Version

3.64.0

Affected Resource(s)/Data Source(s)

azurerm_data_factory_linked_service_azure_databricks

Terraform Configuration Files

resource "azurerm_data_factory_linked_service_azure_databricks" "job_compute_single_node" {
  adb_domain                 = "https://${azurerm_databricks_workspace.this.workspace_url}"
  data_factory_id            = azurerm_data_factory.this.id
  msi_work_space_resource_id = azurerm_databricks_workspace.this.id
  name                       = "job_compute_single_node"
  parameters = {
    "par_cluster_node_type" = ""
    "par_cluster_version"   = ""
  }

  new_cluster_config {
    cluster_version = "@linkedService().par_cluster_version"
    custom_tags = {
      "ResourceClass" = "SingleNode"
    }
    init_scripts          = []
    max_number_of_workers = 0
    min_number_of_workers = 0
    node_type             = "@linkedService().par_cluster_node_type"
    spark_config = {
      "spark.databricks.cluster.profile" = "singleNode"
      "spark.master"                     = "local[*, 4]"
    }
    spark_environment_variables = {
      "PYSPARK_PYTHON" = "/databricks/python3/bin/python3"
    }
  }
}

Debug Output/Panic Output

Error: expected new_cluster_config.0.max_number_of_workers to be in the range (1 - 25000), got 0

Error: expected new_cluster_config.0.min_number_of_workers to be in the range (1 - 25000), got 0

Expected Behaviour

0 worker should be a valid value when it comes to "single node" configurations.

Actual Behaviour

Currently 0 worker is an invalid value.

Steps to Reproduce

No response

Important Factoids

No response

References

No response

[ggmahyar]

We are also very much interested in this one. Probably can be achieved easily by changing the min / max range to (0 - 25000). Using template deployment is ugly and does not allow a clear resource management.