search

Mongey / terraform-provider-kafka

Terraform provider for managing Apache Kafka Topics + ACLs
MIT License
517 stars 129 forks source link

Update Kafka topic returns "error: Empty Summary: This is always a bug in the provider and should be reported to the provider developers." #281

Open christeoh opened 2 years ago

christeoh commented 2 years ago

I'm trying out this module to update Kafka topic configuration (increase number of partitions, change retention period), the terraform plan command works as expected and displays the expected plan. When I run terraform apply it throws the following error:-

error: Empty Summary: This is always a bug in the provider and should be reported to the provider developers.

The target cluster is a Confluent Kafka 6.2.2 cluster if that helps and using v0.5.1 of the provider.

This is in the context of an integration test where I create the topic, which works, attempt to update which fails. I can create and delete topics without error though.

Mongey commented 2 years ago

Can you provide a sample terraform config that reproduces this? Can you provider the output of TF_LOG=1 terraform apply ?

rharriso commented 1 year ago

Seeing this error as well, here is some relevant log output

2022-10-25T11:42:57.765-0500 [TRACE] evalApplyProvisioners: kafka_topic.alerts__delivery__web_alerts is not freshly-created, so no provisioning is required
2022-10-25T11:42:57.765-0500 [TRACE] evalApplyProvisioners: kafka_topic.metrics__gmi is not freshly-created, so no provisioning is required
2022-10-25T11:42:57.765-0500 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for kafka_topic.alerts__delivery__web_alerts
2022-10-25T11:42:57.765-0500 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for kafka_topic.alerts__delivery__web_alerts
2022-10-25T11:42:57.765-0500 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for kafka_topic.metrics__gmi
2022-10-25T11:42:57.765-0500 [TRACE] statemgr.Filesystem: have already backed up original ./stag.tfstate to ./stag.tfstate.backup on a previous write
2022-10-25T11:42:57.765-0500 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for kafka_topic.metrics__gmi
2022-10-25T11:42:57.767-0500 [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 105
2022-10-25T11:42:57.767-0500 [TRACE] statemgr.Filesystem: writing snapshot at ./stag.tfstate
2022-10-25T11:42:57.788-0500 [ERROR] vertex "kafka_topic.alerts__delivery__email_alerts" error: Empty Summary: This is always a bug in the provider and should be reported to the provider developers.
2022-10-25T11:42:57.788-0500 [TRACE] vertex "kafka_topic.alerts__delivery__email_alerts": visit complete, with errors
2022-10-25T11:42:57.788-0500 [ERROR] vertex "kafka_topic.alerts__delivery__email_alerts" error: Empty Summary: This is always a bug in the provider and should be reported to the provider developers.

The updates were successful & the state changes were recorded.

I'd be happy to help debug this, but I'd need some guidance on how to incorporate ad hoc builds in to a terraform workspace. I just started working with terraform last week.

bitdean commented 1 year ago

This problem exists with latest version: 0.5.4, AWS MSK kafka 3.4.0, we are using cdktf to create topics and we set custom config:

kafka_topic.Topic(
      self,
      f"kafka-topic-test",
      name="test"
      partitions=3,
      replication_factor=2,
      config={"min.insync.replicas": "2"},
)

This create topic normally, but on next run when it should refresh state, it says that state vanished for config value:

[WARN]  Provider "registry.terraform.io/mongey/kafka" produced an unexpected new value for kafka_topic.test during refresh.
--
33014 | - .config: element "min.insync.replicas" has vanished

Additional logs:

kafka-acl    # kafka_topic.test (test) will be updated in-place
             ~ resource "kafka_topic" "test" {
                 ~ config             = {
                     + "min.insync.replicas" = "2"
                   }
                   id                 = "test"
                   name               = "test"
                   # (2 unchanged attributes hidden)
               }

kafka-acl  kafka_topic.test (test-topic-test): Modifying... [id=test]
           2023-08-28T16:35:24.408Z [INFO]  Starting apply for kafka_topic.test (test-topic-test)
           2023-08-28T16:35:24.408Z [DEBUG] kafka_topic.test (test-topic-test): applying the planned Update change
           2023-08-28T16:35:24.409Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [TRACE] configuring bootstrap_servers {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.409Z
           2023-08-28T16:35:24.409Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [INFO] configuring kafka client with {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.409Z
           2023-08-28T16:35:24.409Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [WARN] no CA file set skipping: timestamp=2023-08-28T16:35:24.409Z
[2023-08-28T16:35:24.447] [TRACE] default - Terraform CLI state machine event: {"type":"OUTPUT_RECEIVED","output":"2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [DEBUG] Got 4 topics from Kafka: timestamp=2023-08-28T16:35:24.446Z\r\n2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [TRACE] lazy client init %!s(<nil>); config, {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.446Z\r\n"}
[2023-08-28T16:35:24.447] [DEBUG] default - creating terraform log hanlder deploy
[2023-08-28T16:35:24.447] [DEBUG] default - [kafka-acl](deploy): 2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [DEBUG] Got 4 topics from Kafka: timestamp=2023-08-28T16:35:24.446Z
2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [TRACE] lazy client init %!s(<nil>); config, {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.446Z

[2023-08-28T16:35:24.447] [DEBUG] default - Filter undefined applied on line '2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [DEBUG] Got 4 topics from Kafka: timestamp=2023-08-28T16:35:24.446Z
2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [TRACE] lazy client init %!s(<nil>); config, {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.446Z
' with result '2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [DEBUG] Got 4 topics from Kafka: timestamp=2023-08-28T16:35:24.446Z
2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [TRACE] lazy client init %!s(<nil>); config, {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.446Z
'
kafka-acl  2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [DEBUG] Got 4 topics from Kafka: timestamp=2023-08-28T16:35:24.446Z
           2023-08-28T16:35:24.447Z [INFO]  provider.terraform-provider-kafka_v0.5.4: 2023/08/28 16:35:24 [TRACE] lazy client init %!s(<nil>); config, {0x400048e5b8 120   ***** ***** true true acl_manager ***** scram-sha512}: timestamp=2023-08-28T16:35:24.446Z
[2023-08-28T16:35:24.460] [TRACE] default - Terraform CLI state machine event: {"type":"OUTPUT_RECEIVED","output":"2023-08-28T16:35:24.460Z [ERROR] provider.terraform-provider-kafka_v0.5.4: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary=\"Empty Summary: This is always a bug in the provider and should be reported to the provider developers.\" tf_provider_addr=provider tf_rpc=ApplyResourceChange tf_req_id=603db84f-2396-7676-d3e7-d4c51fc4610d @caller=/home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_proto_version=5.3 tf_resource_type=kafka_topic timestamp=2023-08-28T16:35:24.460Z\r\n"}
[2023-08-28T16:35:24.461] [DEBUG] default - creating terraform log hanlder deploy
[2023-08-28T16:35:24.461] [DEBUG] default - [kafka-acl](deploy): 2023-08-28T16:35:24.460Z [ERROR] provider.terraform-provider-kafka_v0.5.4: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="Empty Summary: This is always a bug in the provider and should be reported to the provider developers." tf_provider_addr=provider tf_rpc=ApplyResourceChange tf_req_id=603db84f-2396-7676-d3e7-d4c51fc4610d @caller=/home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_proto_version=5.3 tf_resource_type=kafka_topic timestamp=2023-08-28T16:35:24.460Z

[2023-08-28T16:35:24.461] [DEBUG] default - Filter undefined applied on line '2023-08-28T16:35:24.460Z [ERROR] provider.terraform-provider-kafka_v0.5.4: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="Empty Summary: This is always a bug in the provider and should be reported to the provider developers." tf_provider_addr=provider tf_rpc=ApplyResourceChange tf_req_id=603db84f-2396-7676-d3e7-d4c51fc4610d @caller=/home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_proto_version=5.3 tf_resource_type=kafka_topic timestamp=2023-08-28T16:35:24.460Z
' with result '2023-08-28T16:35:24.460Z [ERROR] provider.terraform-provider-kafka_v0.5.4: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="Empty Summary: This is always a bug in the provider and should be reported to the provider developers." tf_provider_addr=provider tf_rpc=ApplyResourceChange tf_req_id=603db84f-2396-7676-d3e7-d4c51fc4610d @caller=/home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_proto_version=5.3 tf_resource_type=kafka_topic timestamp=2023-08-28T16:35:24.460Z
'
kafka-acl  2023-08-28T16:35:24.460Z [ERROR] provider.terraform-provider-kafka_v0.5.4: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="Empty Summary: This is always a bug in the provider and should be reported to the provider developers." tf_provider_addr=provider tf_rpc=ApplyResourceChange tf_req_id=603db84f-2396-7676-d3e7-d4c51fc4610d @caller=/home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_proto_version=5.3 tf_resource_type=kafka_topic timestamp=2023-08-28T16:35:24.460Z
[2023-08-28T16:35:24.464] [TRACE] default - Terraform CLI state machine event: {"type":"OUTPUT_RECEIVED","output":"2023-08-28T16:35:24.461Z [DEBUG] State storage *remote.State declined to persist a state snapshot\r\n2023-08-28T16:35:24.461Z [ERROR] vertex \"kafka_topic.test\" error: Empty Summary: This is always a bug in the provider and should be reported to the provider developers.\r\n2023-08-28T16:35:24.461Z [DEBUG] states/remote: state read serial is: 4; serial is: 4\r\n2023-08-28T16:35:24.461Z [DEBUG] states/remote: state read lineage is: 07099d25-c798-3eb7-c660-f153a03b768f; lineage is: 07099d25-c798-3eb7-c660-f153a03b768f\r\n"}
[2023-08-28T16:35:24.464] [DEBUG] default - creating terraform log hanlder deploy
[2023-08-28T16:35:24.464] [DEBUG] default - [kafka-acl](deploy): 2023-08-28T16:35:24.461Z [DEBUG] State storage *remote.State declined to persist a state snapshot
2023-08-28T16:35:24.461Z [ERROR] vertex "kafka_topic.test" error: Empty Summary: This is always a bug in the provider and should be reported to the provider developers.
2023-08-28T16:35:24.461Z [DEBUG] states/remote: state read serial is: 4; serial is: 4
2023-08-28T16:35:24.461Z [DEBUG] states/remote: state read lineage is: 07099d25-c798-3eb7-c660-f153a03b768f; lineage is: 07099d25-c798-3eb7-c660-f153a03b768f

This issue can be repeated just by running same config multiple times.

dim-ops commented 16 hours ago

I have the same issue, and none clue in logs. Do you have an idea ? @Mongey