Error: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

[ivialex-mcd]

Hi guys,

I was trying create the topics in AWS MSK Cluster, but show this error below for me.

Error: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

  on main.tf line 33, in resource "kafka_topic" "rfmkafkatopic":
  33: resource "kafka_topic" "rfmkafkatopic" {

Note: I was running terraform plan by GitHub Actions. Do I need to set up some security group for example?

And I was using this module to create MSK cluster

module "kafka" {
  source  = "cloudposse/msk-apache-kafka-cluster/aws"
  version = "0.5.2"

  name                      = random_id.msk_cluster_id.hex
  vpc_id                    = data.aws_vpc.my_vpc.id
  security_groups           = [module.eks.worker_security_group_id, var.bastion_host_security_group_id, var.gh_runner_sg]
  subnet_ids                = data.aws_subnet_ids.my_private_subnets.ids
  kafka_version             = var.kafka_version
  number_of_broker_nodes    = var.number_of_broker_nodes
  broker_instance_type      = var.broker_instance_type
  client_broker             = "TLS_PLAINTEXT"
  client_tls_auth_enabled   = false
  client_sasl_scram_enabled = false
  cloudwatch_logs_log_group = aws_cloudwatch_log_group.msk.name
  cloudwatch_logs_enabled   = true
  jmx_exporter_enabled      = true
  node_exporter_enabled     = true

  context = module.this.context

}

My provider.tf file

provider "kafka" {
  bootstrap_servers = [data.aws_msk_cluster.cluster.bootstrap_brokers_tls]
  skip_tls_verify   = var.msk_skip_tls_verify
  tls_enabled       = var.msk_tls_enabled
}

My versions.tf file

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.36"
    }

    http = {
      source  = "terraform-aws-modules/http"
      version = "2.4.1"
    }

    kafka = {
      source  = "Mongey/kafka"
      version = "0.3.3"
    }

    local    = ">= 1.4"
    null     = ">= 2.1"
    template = ">= 2.1"
    random   = ">= 2.1"
  }
}

my data.tf file

data "aws_msk_cluster" "cluster" {
  cluster_name = local.msk_name
}