Attempting to create topics in a foreach within a resource

[stripthesoul]

Hey, Im using the below code to create topics in our Confluent Cloud envs from a var list.

  for_each = toset( var.tf_var_confluent_cloud_kafka_topic_list )
  kafka_cluster = confluentcloud_kafka_cluster.terraform-cluster.id
  topic_name = format("%s%s","tf-",each.key)
  partitions_count = 7
  http_endpoint = confluentcloud_kafka_cluster.terraform-cluster.http_endpoint
  config = {
    "retention.ms" = "6789000"
  }
  credentials {
    key = var.tf_var_confluent_cloud_kafka_cluster_api_key
    secret = var.tf_var_confluent_cloud_kafka_cluster_api_secret
  }
}

When applying the changes, the plugin stops responding:

╷
│ Error: Request cancelled
│ 
│   with confluentcloud_kafka_topic.all_topics[t"opic_name"],
│   on providers.tf line 25, in resource "confluentcloud_kafka_topic" "all_topics":
│   25: resource "confluentcloud_kafka_topic" "all_topics" {
│ 
│ The plugin.(*GRPCProvider).ValidateResourceConfig request was cancelled.
╵
╷
│ Error: Plugin did not respond
│ 
│   with confluentcloud_kafka_topic.all_topics["topic_name"],
│   on providers.tf line 25, in resource "confluentcloud_kafka_topic" "all_topics":
│   25: resource "confluentcloud_kafka_topic" "all_topics" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.

I also get the following error:

Stack trace from the terraform-provider-confluentcloud_0.5.0 plugin:

panic: reflect: call of reflect.Value.FieldByName on zero Value

goroutine 120 [running]:
reflect.flag.mustBe(...)
    /usr/local/golang/1.16/go/src/reflect/value.go:221
reflect.Value.FieldByName(0x0, 0x0, 0x0, 0x19da363, 0x6, 0x0, 0x140, 0x12b)
    /usr/local/golang/1.16/go/src/reflect/value.go:903 +0x25a
github.com/confluentinc/terraform-provider-ccloud/internal/provider.createDiagnosticsWithDetails(0x1adb6a0, 0xc00082c280, 0xc0003fd5a0, 0x3, 0x3)
    src/github.com/confluentinc/terraform-provider-confluentcloud/internal/provider/utils.go:304 +0x2c5
github.com/confluentinc/terraform-provider-ccloud/internal/provider.kafkaTopicCreate(0x1aea3e8, 0xc0002a0060, 0xc0000f8000, 0x1926000, 0xc000201030, 0xc0007be500, 0x146a3aa, 0xc000402100)
    src/github.com/confluentinc/terraform-provider-confluentcloud/internal/provider/resource_kafka_topic.go:141 +0x525
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0xc0003d8ee0, 0x1aea378, 0xc0003c64c0, 0xc0000f8000, 0x1926000, 0xc000201030, 0x0, 0x0, 0x0)
    pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.10.1/helper/schema/resource.go:341 +0x17f
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc0003d8ee0, 0x1aea378, 0xc0003c64c0, 0xc000838820, 0xc000402100, 0x1926000, 0xc000201030, 0x0, 0x0, 0x0, ...)
    pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.10.1/helper/schema/resource.go:467 +0x67b
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc0000b0108, 0x1aea378, 0xc0003c64c0, 0xc0005c0320, 0x19e3224, 0x12, 0x0)
    pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.10.1/helper/schema/grpc_provider.go:977 +0xacf
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc00060e180, 0x1aea420, 0xc0003c64c0, 0xc0008c8000, 0x0, 0x0, 0x0)
    pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.5.0/tfprotov5/tf5server/server.go:603 +0x465
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x19a12e0, 0xc00060e180, 0x1aea420, 0xc00054a300, 0xc0008a8180, 0x0, 0x1aea420, 0xc00054a300, 0xc000102280, 0x27c)
    pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.5.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:380 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002c6540, 0x1af1b98, 0xc000603800, 0xc00027e200, 0xc00060c570, 0x1f9bb00, 0x0, 0x0, 0x0)
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:1210 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc0002c6540, 0x1af1b98, 0xc000603800, 0xc00027e200, 0x0)
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:1533 +0xd0c
google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000274340, 0xc0002c6540, 0x1af1b98, 0xc000603800, 0xc00027e200)
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:871 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
    pkg/mod/google.golang.org/grpc@v1.33.2/server.go:869 +0x1fd

Error: The terraform-provider-confluentcloud_0.5.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue ```

[linouk23]

👋 @stripthesoul,

Could you downgrade the version of your provider to 0.4.0 and rerun terraform apply? You should be able to see a little bit more descriptive error (this NPE that you can see will be fixed in our next 0.6.0 version)?

[stripthesoul]

Actually reverting to lower version helped! Testing. Thanks @linouk23

[linouk23]

@stripthesoul we're very excited to let you know we've just published a new version of TF Provider that includes a fix for this issue among other very exciting improvements: it enables fully automated provisioning of our key Kafka workflows (see the demo) with no more manual intervention and makes it our biggest and most impactful release.

The only gotcha we've renamed it from confluentinc/confluentcloud to confluentinc/confluent but we published a migration guide so it should be fairly straightforward. The existing confluentinc/confluentcloud will be deprecated soon so we'd recommend switching as soon as possible.

New confluentinc/confluent provider also includes a lot of sample configurations so you won't need to write them from scratch. You can find them here, find a full list of changes here.