search

Azure / terraform-azurerm-openai

Terraform module for deploying Azure OpenAI Service.
MIT License
44 stars 33 forks source link

Support for Dynamic Quota in Chat-Models #91

Open g4challenge opened 3 months ago

g4challenge commented 3 months ago

Is there an existing issue for this?

Description

I would need to enable dynamic quota for the default gpt models

New or Affected Resource(s)/Data Source(s)

openai_deployment

Potential Terraform Configuration

eployment = {                                # TODO make sure to update corresponding litellm config
    "chat_model" = {
      name          = "gpt-4o"
      model_format  = "OpenAI"
      model_name    = "gpt-4o"
      model_version = "2024-05-13"
      scale_type    = "Standard"
      dynamic_quota_enabled = true
      #capacity      = 120
    },

References

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/dynamic-quota

zioproto commented 3 months ago

Hello @g4challenge,

TL;DR waiting on Terraform provider feature

to implement this feature in the module we need first the azurerm provider to support this feature.

I see there is a open issue https://github.com/hashicorp/terraform-provider-azurerm/issues/23988 and an existing PR https://github.com/hashicorp/terraform-provider-azurerm/pull/25401

We have to wait for the feature to be merged.

thanks

zioproto commented 3 months ago

Hello @g4challenge

I also noticed that the feature is in preview. We will add the feature to the module only when it is promoted to GA.