confluentinc / terraform-provider-confluent

Terraform Provider for Confluent
Apache License 2.0
118 stars 61 forks source link

Shared Kafka TF #379

Closed th3cod3r closed 2 months ago

th3cod3r commented 2 months ago

Hi

I am testing shared kafka with the terraform. In my scenario, the kafka server is running on remote server in k8s cluster and i have to just connect with that with a different tenant id. Just FYI, when i connect with the different tenant id the zookeeper will inject some producer which is already drafted (with the TF file the working is same or not?). Also service has been exposed and when i try to connect with a pod it is connected but with the TF not.

when i prepare the TF file with different provider as i unable to understand the confluent provider and resources very well, i have encountered with below error Error: kafka: client has run out of available brokers to talk to: EOF with kafka_acl.tenant1, on main.tf line 38, in resource "kafka_acl" "tenant1": 38: resource "kafka_acl" "tenant1" {

Error: kafka: client has run out of available brokers to talk to: EOF with kafka_acl.tenant2, on main.tf line 47, in resource "kafka_acl" "tenant2": 47: resource "kafka_acl" "tenant2" {

Reqirements: I just only want to connect in it with a specific tenant id by which zookeeper will made separate kafka database with the tenant id (When a another server will connect zookeeper will auto-insert into it with the particular tenant id which i defined). And have to apply some acl for security as well.

My sample TF file as below terraform { required_providers { kafka = { source = "Mongey/kafka" version = "0.7.1" } } }

provider "kafka" { bootstrap_servers = ["192.168.24.15:31000"] }

resource "kafka_topic" "logs" { name = "a" replication_factor = 1 partitions = 1 }

resource "kafka_quota" "quota" { entity_name = "a" entity_type = "user" }

resource "kafka_acl" "test" { resource_name = "a" resource_type = "Topic" acl_principal = "User:test2" acl_host = "192.168.24.15" acl_operation = "All" acl_permission_type = "Allow" }

linouk23 commented 2 months ago

@th3cod3r seems like you are not using TF Provider for Confluent as there's the following snippet in your TF configuration:

terraform {
   required_providers {
      kafka = {
         source = "Mongey/kafka"
         version = "0.7.1"
      }
   }
}

instead of

# Configure the Confluent Provider
terraform {
  required_providers {
    confluent = {
      source  = "confluentinc/confluent"
      version = "1.74.0"
    }
  }
}
th3cod3r commented 1 month ago

Hi @linouk23 , I can change the provider with confluent but i need some guidance to do that as it is completely based on the cloud structure. I already defined my requirements above, if possible could you please help me with my TF file.