Support for provisioned azurerm_cognitive_deployment (and azurerm_cognitive_account) in azurerm provider

[DanHyland]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Description

Per guidance from our TAM, we are managing TPU deployments via azapi, but also are required to allocated TPU capacity to azurerm_cognitive_accounts via the web console.

Concretely we would like the ability to:

• Assign TPU capacity to a azurerm_cognitive_account via Terraform
• Ability to deploy provisoned capacity instances via azurerm_cognitive_deployment resource instead of the azapi example below.

Here is how we manage the model_deployment.

#AzAPI is used to create the deployment so that the PTU size and model versions can be set
resource "azapi_resource" model_deployment" {
  type      = "Microsoft.CognitiveServices/accounts/deployments@2023-05-01"
  name      = "gpt-35-turbo-0301"
  parent_id = azurerm_cognitive_account.account.id
  body = jsonencode({
    sku = {
      name     = "Provisioned", # This specifies a provisioned throughput deployment
      capacity = 300            # This deployment will be sized to 300 PTUs
    },
    properties = {
      model = {
        format  = "OpenAI",
        name    = "gpt-35-turbo",
        version = "0301" # Deploy gpt-35-turbo with version 0301 
      }
    }
  })
  timeouts { # Provisioned deployments take some time to create/update
    create = "1h30m"
    update = "1h30m"
    delete = "1h30m"
  }
  depends_on = [azurerm_cognitive_account.account]
}

New or Affected Resource(s)/Data Source(s)

azurerm_cognitive_deployment, azurerm_cognitive_account

Potential Terraform Configuration

No response

References

No response

[liuwuliuyun]

Hi @DanHyland, thanks for raising this issue. I think AzureRM provider does support deploy provisoned capacity instances via azurerm_cognitive_deployment resource. Could you try following template and let me know if it works?

resource "azurerm_cognitive_deployment" "example" {
  name                 = "example-cd"
  cognitive_account_id = azurerm_cognitive_account.example.id
  model {
    format  = "OpenAI"
    name    = "gpt-35-turbo"
    version = "0301"
  }

  scale {
    type     = "Provisioned"
    capacity = 300
  }
}

[DanHyland]

@liuwuliuyun Fair, our reps pointed only at AZPI for a template. The part i'm more concerned about support is the commitment allocation to Cognitive Accounts. As a pre step we were told to assign commitments to a cognitive account and then models deployment to the account could be allocated TPUs. Is there support for the commitment assignments in TF?

[liuwuliuyun]

Thanks for your reply, you could use AzAPI provider to create commitment plan and do association. We do have plans for supporting commitment plan assignment in AzureRM in the future and I am working with the cognitive service team on this.

[cenhao]

@liuwuliuyun is the type = "Provisioned" supported? Looking at the code it's not one of the supported values https://github.com/hashicorp/terraform-provider-azurerm/blob/main/internal/services/cognitive/cognitive_deployment_resource.go#L143

[liuwuliuyun]

@liuwuliuyun is the type = "Provisioned" supported? Looking at the code it's not one of the supported values https://github.com/hashicorp/terraform-provider-azurerm/blob/main/internal/services/cognitive/cognitive_deployment_resource.go#L143

Hi @cenhao , the code you refer is tier rather than type. You could put any value that azure REST API allows to type value.

[kzap]

When will the "sku" block be supported instead of the "scale" block?

• I see some work done on this here: https://github.com/hashicorp/terraform-provider-azurerm/pull/22223

[liuwuliuyun]

When will the "sku" block be supported instead of the "scale" block?

• I see some work done on this here: azurerm_cognitive_deployment - Upgrade cognitive api version, update resource schema to fix deprecated argument's issue #22223

Hello @kzap , we are adding sku block support to the terraform azurerm provider in version 4.0 and later, for compatibility reasons. However, the scale block is using the same API as the future sku block, so you can use it in the same way.