google_discovery_engine_chat_engine can not be deployed on regional locations

marque2 commented 1 month ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to a user, that user is claiming responsibility for the issue.
Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.8.3 on linux_amd64

provider registry.terraform.io/hashicorp/google v6.5.0
provider registry.terraform.io/hashicorp/google-beta v6.5.0

Affected Resource(s)

google_discovery_engine_chat_engine

Terraform Configuration

resource "google_discovery_engine_data_store" "test" {
  content_config              = "PUBLIC_WEBSITE"
  create_advanced_site_search = true
  data_store_id               = "test"
  display_name                = "test"
  industry_vertical           = "GENERIC"
  location                    = "eu"
  project                     = var.project_id
  solution_types              = ["SOLUTION_TYPE_CHAT"]

  lifecycle {
    # XXX: solution_types currently does not support the SOLUTION_TYPE_GENERATIVE_CHAT value
    #  This value gets added to the solution_types after attaching a generative chat engine to the data store
    #  We need to ignore changes to avoid the removal of generative features until google provider supports it
    ignore_changes = [solution_types]
  }
}

resource "google_discovery_engine_chat_engine" "test" {
  collection_id     = "default_collection"
  data_store_ids    = [google_discovery_engine_data_store.test.data_store_id]
  display_name      = "test"
  engine_id         = "test"
  industry_vertical = "GENERIC"
  location          = "europe-west3"

  chat_engine_config {
    agent_creation_config {
      default_language_code = "de"
      time_zone             = "Europe/Madrid"
    }
  }
}

Debug Output

https://gist.github.com/marque2/df5b06ed20f174dac7cebc9ff4d258fc

Expected Behavior

A VertexAI agent gets created.

Actual Behavior

Terraform raises the following error:

Error: Error creating ChatEngine: googleapi: got HTTP response code 404

More details on the error on the Debug Output.

Steps to reproduce

terraform apply

Important Factoids

Agents can be created VIA TERRAFORM on global and eu regions.
Agents can be created MANUALLY VIA VERTEXAI WEB UI on europe-west3 and other regional locations.

References

No response

b/376524698

ggtisc commented 3 weeks ago

Hi @marque2!

I noticed a mistake on the google_discovery_engine_chat_engine.location you are using the value europe-west3 for the location as if it were a region instead of a location try to assign the value of 'eu' for this value instead of europe-west3 and see if it works. Example:

resource "google_discovery_engine_chat_engine" "test" {
  # some code
  location          = "eu"
  # some code
}

marque2 commented 3 weeks ago

Hi @marque2!

I noticed a mistake on the google_discovery_engine_chat_engine.location you are using the value europe-west3 for the location as if it were a region instead of a location try to assign the value of 'eu' for this value instead of europe-west3 and see if it works. Example:
resource "google_discovery_engine_chat_engine" "test" {
  # some code
  location          = "eu"
  # some code
}

Hi @ggtisc, thanks for taking a look!

I tried it and successfully deployed it. The problem with this solution is that by setting location = "eu" we can not control the region and the agent gets deployed on europe-west1 instead of europe-west3:

The underlying problem of why we need it to be on this particular region is that inside the VertexAI agent we have a connection with a cloud run service deployed on europe-west3 (there is an OpenAPI tool inside the agent with a connection to this cloud run via service directory). This connection only seems to work if both the agent and the cloud run service are in the exact same region:

The agent deployed on europe-west3 manually works perfectly. However, the agent deployed via TF on europe-west1 fails to connect to the cloud run service even when using the exact same configuration (exporting and restoring the agent).

Is there a way to specify the region and not only the location?

Thanks again!

ggtisc commented 3 weeks ago

This issue seems like an enhancement since when reviewing the terraform registry documentation and the API in both, it is explicitly mentioned that the configuration is limited to the global, eu and us locations. There isn't much information beyond the links I share and in addition to the fact that by default it only handles 2 regions (us-central -1 and eu-west-1) limiting the possibilities to scenarios like this, which is why I am forwarding this issue as such

hashicorp / terraform-provider-google