Import Kafka clusters and topics

[losfair]

Hi,

When I attempt to import an existing Kafka cluster (or topic) I see an error:

Error: resource upstash_kafka_topic doesn't support import

It would be useful to support importing existing resources created from the web UI.

[burak-upstash]

Hey, @losfair. We will have a look at this and get back to this issue. Thanks for the suggestion!

[Cliftonz]

@losfair are you saying https://registry.terraform.io/providers/upstash/upstash/latest/docs/data-sources/kafka_cluster_data and https://registry.terraform.io/providers/upstash/upstash/latest/docs/data-sources/kafka_topic_data not working?

Do you have a small example we can look at?

[losfair]

@Cliftonz The data sources are working. The thing that isn't working correctly is using resource for an existing resource:

resource "upstash_kafka_topic" "my-topic" {
  cluster_id       = var.my_cluster_id
  topic_name       = "my-topic"
  partitions       = 1
  retention_size   = -1
  retention_time   = 86400 * 1000 * 7
  max_message_size = 1048576
  cleanup_policy   = "delete"

  lifecycle {
    prevent_destroy = true
  }
}

And import the resource with:

terraform import upstash_kafka_topic.my-topic my-topic

gives the error:

Error: resource upstash_kafka_topic doesn't support import

[burak-upstash]

Hello, we haven't looked into importing yet unfortunately

[burak-upstash]

Hello, @losfair . We have released the new version which has the import capability for all the resources. Closing the issue

[mattfysh]

Was this tested for kafka clusters? I can't seem to get it to work I've reviewed the PR at https://github.com/upstash/terraform-provider-upstash/pull/31

I don't think data.Id() is going to return you anything, because there is no resource state yet before the resource has been imported: https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/schema/resource_data.go#L250-L268

Perhaps you are meant to get the requested ID another way?

[burak-upstash]

Hello @mattfysh What is the error you are getting? You have to define an empty resource, and import your kafka cluster to that empty resource first. After that, if you try to terraform apply, you will see that it will require you to refill that resource definition according to the imported resource.

[mattfysh]

The error im seeing is a 404, could not find Kafka cluster

But when I make the same call manually via curl using the same email and token it works, and I checked the provider code and it seems to be making the same call that I made with curl

I am using the Pulumi package if that helps, could that be causing the issue?

Thanks

On Sun, 7 Jan 2024 at 10:48 pm, burak-upstash @.***> wrote:

Hello @mattfysh https://github.com/mattfysh What is the error you are getting? You have to define an empty resource, and import your kafka cluster to that empty resource first. After that, if you try to terraform apply, you will see that it will require you to refill that resource definition according to the imported resource.

— Reply to this email directly, view it on GitHub https://github.com/upstash/terraform-provider-upstash/issues/25#issuecomment-1880050591, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADSZ4XUZ2MXZFT2HFDIICLYNKKSFAVCNFSM6AAAAAAUQ7SZLCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGA2TANJZGE . You are receiving this because you were mentioned.Message ID: @.***>

[burak-upstash]

Oh, could you open an issue there, then? There might be a chance that pulumi version is not released, that is compatible with latest version of terraform

[binnsr]

This capability is not documented in the v1.5.3 ( https://registry.terraform.io/providers/upstash/upstash/latest/docs ) provider in the Terraform registry. Where can I find documentation to get this working to import kafka clusters and topics between terraform workspaces?