API Key Resource: How to create a cluster & topic?

[vilmosnagy]

I have the following terraform file based on the examples:

variable "env" {
  default = "test"
}

provider "confluentcloud" {
  api_key    = var.confluentcloud_api_key
  api_secret = var.confluentcloud_api_secret
}

variable "confluentcloud_api_key" {}
variable "confluentcloud_api_secret" {}

resource "confluentcloud_environment" "environment" {
  display_name = var.env
}

resource "confluentcloud_kafka_cluster" "basic-cluster" {
  display_name = var.env
  availability = "SINGLE_ZONE"
  cloud        = "AZURE"
  region       = var.region

  basic {

  }

  environment {
    id = confluentcloud_environment.environment.id
  }
}

resource "confluentcloud_kafka_topic" "transit-alert-trip-patches" {
  kafka_cluster      = confluentcloud_kafka_cluster.basic-cluster.id
  topic_name         = "transit-alert.trip-patches"
  partitions_count   = 4
  http_endpoint      = confluentcloud_kafka_cluster.basic-cluster.http_endpoint
  config = {
#    "cleanup.policy"    = "compact"
#    "max.message.bytes" = "12345"
#    "retention.ms"      = "67890"
  }
  credentials {
    key    = "<Kafka API Key for confluentcloud_kafka_cluster.basic-cluster>"
    secret = "<Kafka API Secret for confluentcloud_kafka_cluster.basic-cluster>"
  }
}

What should I put to the confluentcloud_kafka_topic.transit-alert-trip-patches.credentials block? With the value of the var.confluentcloud_api_key & var.confluentcloud_api_secret the terraform apply failes. I see no related output from the confluentcloud_kafka_cluster.basic-cluster block.

Is there a way to:

• create an environment
• create the cluster
• and create the topics in one terraform apply?

Regards,

[linouk23]

Thanks for raising an issue @vilmosnagy!

Reading our sample guide might help to get more context.

More specifically, you for 0.2.0 TF provider version you need to follow this section. The catch is you need to use Cluster API Key instead of Cloud API Key:

credentials {
  key = var.kafka_api_key
  secret = var.kafka_api_secret
}

Let me know if that helps!

and create the topics in one terraform apply?

It's not possible in 0.2.0 version just yet but might become available in 0.3.0 release when we add a new API Key TF resource.

[vilmosnagy]

Thanks four your quick reply @linouk23!

If I understand you correctly, there's no way to create:

• the cluster
• the topics in this cluster

And there's no way to create API keys for the cluster in terraform as well.

in one terraform run. I have to:

• create the cluster with a terraform apply -var="should_create_the_topics_as_well=false"
• go to the confluent cloud website, and create an API key for the cluster on the UI
• rerun my terraform script with terraform apply -var="should_create_the_topics_as_well=true"

Is that correct? That seems a bit complicated, that I can't create my whole infrastructure from terraform in one go, but I have to go to the UI multiple times.

[linouk23]

Is that correct? That seems a bit complicated, that I can't create my whole infrastructure from terraform in one go, but I have to go to the UI multiple times.

That's exactly right!

That said, we are aware of these important issues and will fix in our new versions of TF provider so it'll be possible to create whole infrastructure from terraform in one go without having to go to the UI at all.

[phumberdroz]

Where can I keep track of this feature request?

[linouk23]

@phumberdroz that's a great question, I'm not sure why I closed this issue.

The best way is probably to start watching this repository and so you're notified whenever there are any updates.

[itaykat]

Does version 0.3.0 fixed that issue? didn't see any related line in the changelog...

[linouk23]

@itaykat it didn't, we're working on a fix that we are hoping to release toward the end of March.

[ConfluentSpencer]

Duplicate of this request: https://github.com/confluentinc/terraform-provider-confluentcloud/issues/4

[linouk23]

@vilmosnagy, @itaykat we're very excited to let you know we've just published a new version of TF Provider that includes api_key resource among other very exciting improvements: it enables fully automated provisioning of our key Kafka workflows (see the demo) with no more manual intervention and makes it our biggest and most impactful release.

The only gotcha we've renamed it from confluentinc/confluentcloud to confluentinc/confluent but we published a migration guide so it should be fairly straightforward. The existing confluentinc/confluentcloud will be deprecated soon so we'd recommend switching as soon as possible.

New confluentinc/confluent provider also includes a lot of sample configurations so you won't need to write them from scratch. You can find them here, find a full list of changes here.