search

cloudposse / terraform-aws-msk-apache-kafka-cluster

Terraform module to provision AWS MSK
https://cloudposse.com/accelerate
Apache License 2.0
73 stars 102 forks source link

Outputs of broker lists creating perpetual diff on MSK clusters with more than 1 broker per AZ #120

Open iftachbyaf opened 5 months ago

iftachbyaf commented 5 months ago

Describe the Bug

On versions 1.2.0/ 1.3.0 / 1.4.0. Newer versions introduce breaking changes so we have yet to upgrade.

Using outputs such as module.msk-apache-kafka-cluster.bootstrap_brokers_tls, we are seeing a perpetual diff on TF runs on MSK clusters with more than 1 broker per AZ.

~ bootstrap_brokers_tls_list = [

It seems that every TF run randomly selects 3 brokers for this output (and other similar broker list outputs), which creates a diff on each TF run.

Is this expected behaviour? As it creates cascading diffs on environments using this output. Is there a workaround to have this perpetual diff stop happening?

Expected Behavior

Ideally I assume we would want all brokers to appear in the output, or at least get the first 3 brokers so we would not have a perpetual diff

Steps to Reproduce

Creates a cluster with more than 3 brokers / with 3 AZs and more than 1 broker per AZ, and set this output.

Screenshots

No response

Environment

No response

Additional Context

No response

iftachbyaf commented 5 months ago

On further research this doesnt seem to be a bug with Cloudposse msk module, seems to be how MSK cluster data source behaves as well...

Has anyone faced this 'issue' and found some solution?