Provider doesn't detect missing ACLs

[endofcake]

I'm using provider 0.2.3 (happens in 0.2.4 too though). Some ACLs in my cluster have disappeared (I verified this by running kafka-acls --list --principal "User:my-test-user". When I re-run Terraform, the provider doesn't detect any changes. I then tried to comment out the configuration and run Terraform again. The plan states that the ACLs will be deleted, but during apply Terraform fails:

kafka_acl.my_test_user_cluster_describe: Destroying... [id=User:my-test-user|*|Describe|Allow|Cluster|kafka-cluster|Literal]
kafka_acl.my_test_user_group_describe: Destroying... [id=User:my-test-user|*|Describe|Allow|Group|*|Literal]
kafka_acl.my_test_user_topic_describe: Destroying... [id=User:my-test-user|*|Describe|Allow|Topic|*|Literal]

Error: There were no acls matching this filter
Error: There were no acls matching this filter
Error: There were no acls matching this filter

Looks like the provider only checks the state file during the plan step, while I would expect it to describe the actual state of the cluster. Potentially similar to https://github.com/Mongey/terraform-provider-kafka/issues/61.

[wuangq]

Did basically same thing, seeing

kafka_acl.payment_payout_describe: Destroying... [id=payment_payout]
kafka_acl.payment_payout_write: Destroying... [id=payment_payout]
kafka_acl.payment_service_group: Destroying... [id=payment-service]
kafka_acl.payment_payout_read: Destroying... [id=payment_payout]
kafka_acl.payment_stripe_write: Destroying... [id=payment_stripe]
kafka_acl.payment_stripe_describe: Destroying... [id=payment_stripe]
kafka_acl.payment_stripe_read: Destroying... [id=payment_stripe]

Error: Unknown operation: 

Error: Unknown operation: 

Error: Unknown operation: 

Error: Unknown operation: 

Error: Unknown operation: 

Error: Unknown operation: 

Error: Unknown operation: 

[Constantin07]

Facing the same issue when I try to destroy topic with ACLs.

kafka_acl.topic_acl[0]: Destruction complete after 0s
kafka_acl.group_acl[1]: Destruction complete after 0s
kafka_acl.topic_acl[1]: Destruction complete after 0s
kafka_acl.group_acl[2]: Destruction complete after 0s
kafka_acl.group_acl[0]: Destruction complete after 0s
kafka_topic.this[0]: Destruction complete after 3s

Error: There were no acls matching this filter

TF provider version 0.2.4, TF 0.12.21, Kafka version 2.2.1

[Mongey]

This should be fixed in 0.2.5

> cd examples && terraform apply

root@bfe37689dc46:/# kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper:2181 --remove --deny-principal User:Alice  --operation Write --topic syslog
Are you sure you want to remove ACLs:
        (principal=User:Alice, host=*, operation=WRITE, permissionType=DENY)
 from resource filter `ResourcePattern(resourceType=TOPIC, name=syslog, patternType=LITERAL)`? (y/n)
y

❯ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

kafka_acl.global: Refreshing state... [id=User:*|*|All|Allow|Topic|*|Literal]
kafka_acl.test: Refreshing state... [id=User:Alice|*|Write|Deny|Topic|syslog|Literal]
kafka_topic.syslog: Refreshing state... [id=syslog]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kafka_acl.test will be created
  + resource "kafka_acl" "test" {
      + acl_host                     = "*"
      + acl_operation                = "Write"
      + acl_permission_type          = "Deny"
      + acl_principal                = "User:Alice"
      + id                           = (known after apply)
      + resource_name                = "syslog"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

❯ terraform apply
kafka_acl.global: Refreshing state... [id=User:*|*|All|Allow|Topic|*|Literal]
kafka_acl.test: Refreshing state... [id=User:Alice|*|Write|Deny|Topic|syslog|Literal]
kafka_topic.syslog: Refreshing state... [id=syslog]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kafka_acl.test will be created
  + resource "kafka_acl" "test" {
      + acl_host                     = "*"
      + acl_operation                = "Write"
      + acl_permission_type          = "Deny"
      + acl_principal                = "User:Alice"
      + id                           = (known after apply)
      + resource_name                = "syslog"
      + resource_pattern_type_filter = "Literal"
      + resource_type                = "Topic"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

kafka_acl.test: Creating...
kafka_acl.test: Creation complete after 0s [id=User:Alice|*|Write|Deny|Topic|syslog|Literal]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.=