Add support for spark_pool_version 3.4 to azurerm_synapse_spark_pool

daniel-brown-improving commented 6 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.7.1

AzureRM Provider Version

3.39.0

Affected Resource(s)/Data Source(s)

azurerm_synapse_spark_pool

Terraform Configuration Files

resource "azurerm_resource_group" "test" {
  name     = "rg-test"
  location = "Central US"
}

resource "azurerm_storage_account" "test" {
  name                            = "test20230104"
  resource_group_name             = azurerm_resource_group.test.name
  location                        = azurerm_resource_group.test.location
  account_tier                    = "Standard"
  account_replication_type        = "ZRS"
  account_kind                    = "StorageV2"
  default_to_oauth_authentication = true
  enable_https_traffic_only       = true
  is_hns_enabled                  = true
  min_tls_version                 = "TLS1_2"
  public_network_access_enabled   = true
}

resource "azurerm_storage_data_lake_gen2_filesystem" "test" {
  name               = "synapse"
  storage_account_id = azurerm_storage_account.test.id
}

resource "azurerm_synapse_workspace" "test" {
  name                                 = "test-workspace"
  resource_group_name                  = azurerm_resource_group.test.name
  location                             = azurerm_resource_group.test.location
  managed_resource_group_name          = "${azurerm_resource_group.test.name}-synapse-managed"
  managed_virtual_network_enabled      = true
  sql_administrator_login              = "test"
  sql_administrator_login_password     = "test"
  storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.test.id

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_synapse_spark_pool" "synapse" {
  name                 = "testpool"
  synapse_workspace_id = azurerm_synapse_workspace.synapse.id
  node_size_family     = "MemoryOptimized"
  node_size            = "Small"
  spark_version        = "3.4"
}

Debug Output/Panic Output

Error: expected spark_version to be one of ["2.4" "3.1" "3.2" "3.3"], got 3.4
│ 
│   with azurerm_synapse_spark_pool.synapse_sp,
│   on synapse-spark-pool.tf line 45, in resource "azurerm_synapse_spark_pool" "synapse_sp":
│    7:   spark_version        = 3.4

Expected Behaviour

It should allow 3.4 for spark_version because it's allowed in the Azure Portal UI.

Actual Behaviour

Doesn't allow the latest version for spark_version (3.4)

Steps to Reproduce

No response

Important Factoids

No response

References

Previous Upgrade: https://github.com/hashicorp/terraform-provider-azurerm/pull/19866

Microsoft Documentation: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-34-runtime

NOTE: it says that 3.4 is in public preview in the docs, but it's actually in GA in the portal. Microsoft hasn't updated their docs!

daniel-brown-improving commented 6 months ago

I created a PR for this: https://github.com/hashicorp/terraform-provider-azurerm/pull/25319

github-actions[bot] commented 5 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-azurerm