Failed to read deployment traffic filter ruleset after create

mpresling commented 1 month ago

Readiness Checklist

[x] I am running the latest version
[x] I checked the documentation and found no answer
[x] I checked to make sure that this issue has not already been filed
[x] I am reporting the issue to the correct repository (for multi-repository projects)

Expected Behavior

Creating a new Traffic Filter with ec_deployment_traffic_filter completes successfully.

Current Behavior

After creating the resource in my account in EC (which happens successfully and it's visible in the console) the provider is unable to read back the resource details/id to store the state.

ec_deployment_traffic_filter.argonaut: Creating...
ec_deployment_traffic_filter.azure: Creating...

│ Error: Failed to read deployment traffic filter ruleset after create.
│ 
│   with ec_deployment_traffic_filter.argonaut,
│   on main.tf line 85, in resource "ec_deployment_traffic_filter" "argonaut":
│   85: resource "ec_deployment_traffic_filter" "argonaut" {
│ 
│ Failed to read deployment traffic filter ruleset after create.
╵
╷
│ Error: Failed to read deployment traffic filter ruleset after create.
│ 
│   with ec_deployment_traffic_filter.azure,
│   on main.tf line 106, in resource "ec_deployment_traffic_filter" "azure":
│  106: resource "ec_deployment_traffic_filter" "azure" {
│ 
│ Failed to read deployment traffic filter ruleset after create.
╵

I also cannot import an existing resource (note: portion of ID replaced with xxx):

    {
      "id" : "8a3910xxx999327",
      "name" : "Argonaut pre",
      "type" : "ip",
      "include_by_default" : false,
      "region" : "azure-australiaeast",
      "rules" : [
        {
          "id" : "xxx",
          "source" : "xxx.xxx.xxx.xxx"
        }
      ],
      "associations" : [
        {
          "entity_type" : "deployment",
          "id" : "xxx"
        }
      ],
      "total_associations" : 1
    }

% terraform import ec_deployment_traffic_filter.argonaut 8a3910xxx999327
data.ec_stack.latest: Reading...
data.ec_stack.latest: Read complete after 1s [id=8.15.1]
ec_deployment_traffic_filter.argonaut: Importing from ID "8a3910xxx999327"...
ec_deployment_traffic_filter.argonaut: Import prepared!
  Prepared ec_deployment_traffic_filter for import
ec_deployment_traffic_filter.argonaut: Refreshing state... [id=8a3910xxx999327]
╷
│ Error: Cannot import non-existent remote object
│ 
│ While attempting to import an existing object to "ec_deployment_traffic_filter.argonaut", the provider detected that no object exists with the given id. Only pre-existing objects can be imported; check that the id is correct and that it is associated with the
│ provider's configured region or endpoint, or use "terraform apply" to create a new remote object for this resource.
╵

Terraform definition

Simple IP filter:

resource "ec_deployment_traffic_filter" "argonaut" {
  name   = "Argonaut ${var.ENV}"
  region = ec_deployment.nzbn.region
  type   = "ip"

  rule {
    source = "xxx.xxx.xxx.xxx"
  }
  lifecycle {
    ignore_changes = [rule]
  }

  depends_on = [ec_deployment.nzbn]
}

Azure Private Link filter:

# https://www.elastic.co/guide/en/cloud/current/ec-traffic-filtering-vnet.html
resource "ec_deployment_traffic_filter" "azure" {
  name   = "NZBN ${var.ENV} Private Link"
  region = data.ec_stack.latest.region
  type   = "azure_private_endpoint"

  rule {
    azure_endpoint_name = var.AZURE_PRIVATE_ENDPOINT_NAME
    azure_endpoint_guid = var.AZURE_PRIVATE_ENDPOINT_ID
  }

  depends_on = [ec_deployment.nzbn]
}

Steps to Reproduce

Create Azure Native ISV resource
Generate Org API key
Run Terraform pipeline to manage deployment (update topology and resource sizing)

Note: other changes such as ec_deployment complete successfully prior to the traffic filters failing.

Context

This used to work and I was able to build the lower environments fine. Then after EC 8.15.0 was released this started failing and I can no longer manage the deployment fully via Terraform.

After upgrading to elastic/ec 0.11.0 it resolved some other problems with clusters.cluster_settings_change_prohibited: The requested cluster metadata changes are not permitted, but not this one.

Your Environment

Version used: ec 0.11.0 / Terraform 1.9.5
Running against Elastic Cloud 8.15.0 & 8.15.1
Operating System and version: Linux (ADO pipeline agent) and macOS Sonoma 14.6.1

argomark commented 1 month ago

Has anyone else seen or heard of this issue? Could it just be the way I'm using it? Any ideas?

Note: argomark == mpresling 😀

lauren-hinge commented 1 month ago

Running into this too. Seems like there is no way to use ec_deployment_traffic_filter via terraform.

Between this and the lack of support for customer-managed keys via this provider, I'm having trouble coming up with any sensible workflow for working with elastic from code.

elastic / terraform-provider-ec