BnMcG
commented
3 years ago Initial testing suggests that downgrading to 0.2.11 (the version in the example) fixes this problem, but upgrading to 0.3.2 does not.
Open BnMcG opened 3 years ago
BnMcG
commented
3 years ago Initial testing suggests that downgrading to 0.2.11 (the version in the example) fixes this problem, but upgrading to 0.3.2 does not.
Mongey
commented
3 years ago Thanks for reporting @BnMcG
This is actually more of an issue with terraform-provider-kafka. I fixed the crash, but actually didn't fix the problem with supporting lazy-provider initiation😅
BnMcG
commented
3 years ago No problem! I suspected that might be the case, but wasn't 100% sure. I'll test 0.3.3 now - or is lazy-provider initiation required? Sorry - I'm not too familiar with how Terraform providers work!
Mongey
commented
3 years ago lazy initiation is needed to run provision the cluster and topic in one run, but, if break it into two different runs ... it should work.
main.tf
terraform {
required_providers {
confluentcloud = {
source = "Mongey/confluentcloud"
}
kafka = {
source = "Mongey/kafka"
version = "0.3.1"
}
}
}
resource "confluentcloud_environment" "environment" {
name = var.confluent_cloud_environment_name
}
resource "confluentcloud_kafka_cluster" "kafka_cluster" {
name = "foo"
service_provider = "aws"
region = "eu-west-2"
availability = "LOW"
environment_id = confluentcloud_environment.environment.id
deployment = {
sku = "BASIC"
}
network_egress = 100
network_ingress = 100
storage = 5000
}
resource "confluentcloud_api_key" "credentials" {
cluster_id = confluentcloud_kafka_cluster.kafka_cluster.id
environment_id = confluentcloud_environment.environment.id
}
terraform apply
Add back in the topic config
main.tf
terraform {
required_providers {
confluentcloud = {
source = "Mongey/confluentcloud"
}
kafka = {
source = "Mongey/kafka"
version = "0.3.1"
}
}
}
resource "confluentcloud_environment" "environment" {
name = var.confluent_cloud_environment_name
}
resource "confluentcloud_kafka_cluster" "kafka_cluster" {
name = "foo"
service_provider = "aws"
region = "eu-west-2"
availability = "LOW"
environment_id = confluentcloud_environment.environment.id
deployment = {
sku = "BASIC"
}
network_egress = 100
network_ingress = 100
storage = 5000
}
resource "confluentcloud_api_key" "credentials" {
cluster_id = confluentcloud_kafka_cluster.kafka_cluster.id
environment_id = confluentcloud_environment.environment.id
}
locals {
bootstrap_servers = [replace(confluentcloud_kafka_cluster.kafka_cluster.bootstrap_servers, "SASL_SSL://", "")]
}
provider "kafka" {
bootstrap_servers = local.bootstrap_servers
tls_enabled = true
sasl_username = confluentcloud_api_key.credentials.key
sasl_password = confluentcloud_api_key.credentials.secret
sasl_mechanism = "plain"
timeout = 10
}
resource "kafka_topic" "bar" {
name = "bar"
replication_factor = 3
partitions = 1
config = {
"cleanup.policy" = "delete"
"retention.ms" = 900000
"retention.bytes" = 10485760
}
}terraform applyI'll work on fixing the lazy initiation over in terraform-provider-kafka
BnMcG
commented
3 years ago I'm happy to have a crack at the lazy initiation if you have any pointers - I googled but nothing too comprehensive came up.
Hi,
I was giving this plugin a try this evening, but unfortunately the example in the README seems to crash with new versions of the Kafka provider:
When I run this on Terraform Cloud:
I'm assuming this is something to do with the Confluent Cloud cluster not existing yet when the Kafka adapter attempts to connect to it, but I'm unsure of how to proceed. Do you have any pointers?
I'd like to be able to create a cluster and some topics in one Terraform plan, if possible.
Cheers.