ACL creation crash on both 0.5.0 and 0.4.0

[stripthesoul]

I am attempting to create an ACL as follows (result from the plan):

  + create

Terraform will perform the following actions:

  # confluentcloud_kafka_acl.terraform-cluster-acl-create will be created
  + resource "confluentcloud_kafka_acl" "terraform-cluster-acl-create" {
      + host          = "*"
      + http_endpoint = "https://pkc-xxxxxxy.us-east-1.aws.confluent.cloud:443"
      + id            = (known after apply)
      + kafka_cluster = "lkc-xxxxxx"
      + operation     = "CREATE"
      + pattern_type  = "LITERAL"
      + permission    = "ALLOW"
      + principal     = "User:sa-12p7n5"
      + resource_name = "kafka-cluster"
      + resource_type = "CLUSTER"

      + credentials {
          + key    = (sensitive value)
          + secret = (sensitive value)
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

when I attempt to create the ACL using confluent cloud provider 0.5.0, I get the following issue:

│ Error: Plugin did not respond
│ 
│   with confluentcloud_kafka_acl.terraform-cluster-acl-create,
│   on providers.tf line 22, in resource "confluentcloud_kafka_acl" "terraform-cluster-acl-create":
│   22: resource "confluentcloud_kafka_acl" "terraform-cluster-acl-create" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.

Stack trace from the terraform-provider-confluentcloud_0.5.0 plugin:

panic: reflect: call of reflect.Value.FieldByName on zero Value

goroutine 41 [running]:
reflect.flag.mustBe(...)
    /usr/local/golang/1.16/go/src/reflect/value.go:221
reflect.Value.FieldByName(0x0, 0x0, 0x0, 0x19da363, 0x6, 0x0, 0x140, 0x12c)
    /usr/local/golang/1.16/go/src/reflect/value.go:903 +0x25a
github.com/confluentinc/terraform-provider-ccloud/internal/provider.createDiagnosticsWithDetails(0x1adb6a0, 0xc00038c500, 0xc000207470, 0x3, 0x3)
    src/github.com/confluentinc/terraform-provider-confluentcloud/internal/provider/utils.go:304 +0x2c5
github.com/confluentinc/terraform-provider-ccloud/internal/provider.kafkaAclCreate(0x1aea3e8, 0xc00054d860, 0xc000228b00, 0x1926000, 0xc0001cd110, 0xc00022f630, 0x146a3aa, 0xc000228980)
    src/github.com/confluentinc/terraform-provider-confluentcloud/internal/provider/resource_kafka_acl.go:179 +0x547
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0xc00042f0a0, 0x1aea378, 0xc000617840, 0xc000228b00, 0x1926000, 0xc0001cd110, 0x0, 0x0, 0x0)
    pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.10.1/helper/schema/resource.go:341 +0x17f
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc00042f0a0, 0x1aea378, 0xc000617840, 0xc00057bc70, 0xc000228980, 0x1926000, 0xc0001cd110, 0x0, 0x0, 0x0, ...)
    pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.10.1/helper/schema/resource.go:467 +0x67b
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc000392108, 0x1aea378, 0xc000617840, 0xc000396d20, 0x19e3224, 0x12, 0x0)
    pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.10.1/helper/schema/grpc_provider.go:977 +0xacf
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc000784200, 0x1aea420, 0xc000617840, 0xc0001e0cb0, 0x0, 0x0, 0x0)
    pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.5.0/tfprotov5/tf5server/server.go:603 +0x465
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x19a12e0, 0xc000784200, 0x1aea420, 0xc000197260, 0xc00054cf60, 0x0, 0x1aea420, 0xc000197260, 0xc00055cc00, 0x2f6)
    pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.5.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:380 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000298540, 0x1af1b98, 0xc0000a3500, 0xc0003fe400, 0xc0003834d0, 0x1f9bb00, 0x0, 0x0, 0x0)
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:1210 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc000298540, 0x1af1b98, 0xc0000a3500, 0xc0003fe400, 0x0)
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:1533 +0xd0c
google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000036250, 0xc000298540, 0x1af1b98, 0xc0000a3500, 0xc0003fe400)
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:871 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:869 +0x1fd

Error: The terraform-provider-confluentcloud_0.5.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

2022-05-03T12:52:53.342-0400 [DEBUG] provider: plugin exited

When I attempt to apply using 0.4.0:

-----------------------------------------------------
2022-05-03T12:55:48.265-0400 [DEBUG] [aws-sdk-go] {}
╷
│ Error: 403 Forbidden
│ 
│   with confluentcloud_kafka_acl.terraform-cluster-acl-create,
│   on providers.tf line 22, in resource "confluentcloud_kafka_acl" "terraform-cluster-acl-create":
│   22: resource "confluentcloud_kafka_acl" "terraform-cluster-acl-create" {
│ 
╵

[danidr7]

Same problem here. A little more details in 0.5.0:

module.confluent_sa.confluentcloud_kafka_acl.all_access_cluster: Creating...
2022-05-03T17:01:17.008-0300 [INFO]  Starting apply for module.confluent_sa.confluentcloud_kafka_acl.all_access_cluster
2022-05-03T17:01:17.008-0300 [DEBUG] module.confluent_sa.confluentcloud_kafka_acl.all_access_cluster: applying the planned Create change
2022-05-03T17:01:17.008-0300 [DEBUG] provider.terraform-provider-confluentcloud_0.5.0: 2022/05/03 17:01:17 [DEBUG] GET https://api.confluent.cloud/service_accounts
2022-05-03T17:01:23.173-0300 [DEBUG] provider.terraform-provider-confluentcloud_0.5.0: 2022/05/03 17:01:23 [DEBUG] POST https://pkc-abcde1.us-east-2.aws.confluent.cloud:443/kafka/v3/clusters/my-cluster-id/acls
2022-05-03T17:01:24.061-0300 [INFO]  provider.terraform-provider-confluentcloud_0.5.0: 2022/05/03 17:01:24 [ERROR] Kafka ACL create failed {CLUSTER kafka-cluster LITERAL User:123456 * ALL ALLOW}, &{403 Forbidden 403 HTTP/2.0 2 0 map[Content-Type:[application/json] Date:[Tue, 03 May 2022 20:01:23 GMT]] {0x140004ba580} -1 [] false false map[] 0x140001b8700 0x1400041cd10}, 403 Forbidden: timestamp=2022-05-03T17:01:24.060-0300
2022-05-03T17:01:24.063-0300 [DEBUG] provider.terraform-provider-confluentcloud_0.5.0: panic: reflect: call of reflect.Value.FieldByName on zero Value

[danidr7]

Actually, I commit a huge mistake, I was confusing cloud API keys(the same configured in provider) with kafka api keys. The resource confluentcloud_kafka_acl requires a kafka api key with ALTER permission in cluster. Reference: https://docs.confluent.io/cloud/current/access-management/authenticate/api-keys/api-keys.html

[stripthesoul]

I used Kafka keys not cloud keys, so my issue still persists

[danidr7]

@stripthesoul Is your Kafka key configured with ALTER permission for the cluster?

[linouk23]

It might be useful to check that the owner of Kafka API Key (service account) has a corresponding role (CloudClusterAdmin) / ACLs assigned.

[stripthesoul]

@linouk23 the owner does have roles, but I will double check and get back to you! Thanks!

[linouk23]

@stripthesoul we're very excited to let you know we've just published a new version of TF Provider that includes a lot of very very exciting improvements: it enables fully automated provisioning of our key Kafka workflows (see the demo) with no more manual intervention and makes it our biggest and most impactful release.

The only gotcha we've renamed it from confluentinc/confluentcloud to confluentinc/confluent but we published a migration guide so it should be fairly straightforward. The existing confluentinc/confluentcloud will be deprecated soon so we'd recommend switching as soon as possible.

New confluentinc/confluent provider also includes a lot of sample configurations so you won't need to write them from scratch. You can find them here, find a full list of changes here.

To be more specific, here's the example that shows how to setup ACLs & service accounts that might help.

Let us know if it helps!

[stripthesoul]

@linouk23 I will test the changes this week and report back. Thank you!

[jorgenfries]

@linouk23 - Seems to be some issues with the new provider - the reference is returning 404 confluentinc/terraform-provider-confluent#11

[danidr7]

@linouk23 I'm very interested in the new solution, however the links you pasted here are broken: https://github.com/confluentinc/terraform-provider-confluent

[linouk23]

@jorgenfries @danidr7 @stripthesoul thanks for reporting the issue!

We've just fixed it, could you try again?