Terraform Aqua provider doesn't properly destroy "aquasec_enforcer_groups"

[ruzickap]

Terraform Aqua provider doesn't properly destroy "aquasec_enforcer_groups".

This issue should be fixed here https://github.com/aquasecurity/terraform-provider-aquasec/issues/135, but it wasn't. (this time I updated the aqusec provider to latest v0.8.12 to verify it)

Terraform Version

❯ terraform -v
Terraform v1.2.2
on darwin_amd64
+ provider registry.terraform.io/aquasecurity/aquasec v0.8.12

Affected Resource(s)

• aquasec_enforcer_groups

Steps to Reproduce

I'm using this code to create enforcer groups in Aqua:

terraform {
  backend "local" {
    path = "terraform.tfstate"
  }
  required_providers {
    aquasec = {
      version = "0.8.12"
      source  = "aquasecurity/aquasec"
    }
  }
}

provider "aquasec" {
  username = var.aquasec_username
  aqua_url = var.aquasec_url
  password = var.aquasec_password
}

variable "aquasec_username" {
  type        = string
  description = "Aqua Username"
}

variable "aquasec_password" {
  type        = string
  description = "Aqua Password"
}

variable "aquasec_url" {
  type        = string
  description = "Aqua URL"
}

resource "aquasec_enforcer_groups" "auqa-enforcers-agent-kubernetes" {
  group_id    = "tes-enforcer"
  type        = "agent"
  description = "Test Enforcer"
  enforce     = true
  # Host Assurance
  host_assurance = true
  # Network Firewall (Host Protection)
  host_network_protection = true
  # Runtime Controls
  host_protection = true
  # Network Firewall (Container Protection)
  network_protection = true
  # Advanced Malware Protection (Container Protection)
  container_antivirus_protection = true
  # Runtime Controls
  container_activity_protection = true
  # Image Assurance
  image_assurance = true
  # Advanced Malware Protection (Host Protection)
  antivirus_protection = true
  # Host Images
  sync_host_images = true
  # Risk Explorer
  risk_explorer_auto_discovery = true

  orchestrator {
    type      = "kubernetes"
    namespace = "aqua"
  }
}

resource "aquasec_enforcer_groups" "auqa-enforcers-kube_enforcers-kubernetes" {
  group_id    = "test-kube_enforcer"
  type        = "kube_enforcer"
  description = "Test Kube Enforcer"
  enforce     = true
  # Enable admission control
  admission_control = true
  # Perform admission control if not connected to a gateway
  block_admission_control = true
  # Enable workload discovery
  auto_discovery_enabled = true
  # Register discovered pod images
  auto_scan_discovered_images_running_containers = true
  # Add discovered registries
  auto_discover_configure_registries = true
  # Kube-bench image path
  kube_bench_image_name = "registry.aquasec.com/kube-bench:v0.6.5"
  # Secret that holds the registry credentials for the Pod Enforcer and kube-bench
  micro_enforcer_secrets_name = "aqua-registry"
  # Auto copy these secrets to the Pod Enforcer namespace and container
  auto_copy_secrets = true

  orchestrator {
    type      = "kubernetes"
    namespace = "aqua"
  }
}

Install kube-enforcer to k8s cluster:

helm repo add aqua-helm https://helm.aquasec.com
helm upgrade --install --version "2022.4.5" --namespace aqua --create-namespace --values - kube-enforcer aqua-helm/kube-enforcer << EOF
global:
  enforcer:
    enabled: true
  gateway:
    address: 25dxxxxxxxx-gw.cloud.aquasec.com
    port: 443
  imageCredentials:
  imageCredentials:
    create: true
    #---# Aqua repository example:
    username: "xxxxxxxxxxxxxx"
    password: "xxxxxxxxxxxxxx"
serviceAccount:
  create: true
clusterName: "kube2.k8s.mylabs.com"
logicalName: "kube2.k8s.mylabs.com"
certsSecret:
  autoGenerate: true
aquaSecret:
  #---# The following value needs to be changed !
  kubeEnforcerToken: "xxxxxxx"
enforcer:
  #---# The following value needs to be changed !
  enforcerToken: "xxxxxxxxxxx"
  logicalName: kube2.k8s.mylabs.com
  nodeName: kube2.k8s.mylabs.com
EOF

Once the enforcer groups are create I connected my k8s cluster and everything is working like expected:

The problem appeared when I tried to run terraform destroy where I'm getting:

aquasec_enforcer_groups.auqa-enforcers-kube_enforcers-kubernetes: Destroying... [id=test-kube_enforcer]
aquasec_enforcer_groups.auqa-enforcers-agent-kubernetes: Destroying... [id=tes-enforcer]
╷
│ Error: failed deleting enforcer group, status code: 400
│
│
╵
╷
│ Error: failed deleting enforcer group, status code: 400

When I delete the k8s cluster I can not destroy it either (see "red" disconnected enforcers):

I need to manually delete connections/enforcers in Aqua GUI and then I can destroy it:

-> Terraform should be able to destroy enforcer groups without any manual intervention.

---

Use case - we are creating / deleting k8s clusters for testing purposes therefore we would like to automate the whole process:

• create cluster
• register cluster to Aqua
• run tests
• delete cluster
• delete cluster form Aqua

[semyonmor]

Hi @ruzickap Sorry, but the updated version for 2022.4 with this fix was not released yet It is planned to be released this week. We will update you when it will be ready in this issue

[ruzickap]

Thank you for explanation. Can we keep this opened till fix for 2022.04 will be released please? Once the fix for Aqua 2022.04 will be released - I'll give it a try and update/close the issue.

[semyonmor]

@ruzickap Of course.

[semyonmor]

Hey @ruzickap The 2022.4, update 4 was released today

[ruzickap]

Thank you... I'll test it next week :-)

[ruzickap]

This is now working fine....

I was able to remove enforcer with "registered k8s cluster" without any issues.

Thank you for fixing it :-)

[semyonmor]

@ruzickap thank you for testing it