confluentinc / terraform-provider-confluent

Terraform Provider for Confluent
Apache License 2.0
30 stars 64 forks source link

Uploading a custom connector to AZURE returns a 400 bad request error #481

Open cconnorsada opened 5 hours ago

cconnorsada commented 5 hours ago

Hello, I'm trying to upload a custom connector to AZURE to use in my Confluent Cloud Kafka cluster.

I was able to upload the connector successfully without issue using the default (AWS). However when I use the cloud input variable to the resource with the AZURE value I get this in return.

╷
│ Error: error creating Custom Connector Plugin "Debezium MongoDB CDC Connector Plugin": 400 Bad Request: error creating Connect plugin
│ 
│   with module.confluent-kafka.confluent_custom_connector_plugin.debezium_mongo,
│   on ../../../../../modules/confluent-kafka/connectors.tf line 12, in resource "confluent_custom_connector_plugin" "debezium_mongo":
│   12: resource "confluent_custom_connector_plugin" "debezium_mongo" {
│ 

Unfortunately because my Kafka cluster is in Azure it also requires the custom connector to be uploaded in Azure. I don't see any other input variables required when using Azure so I'm not sure why it's giving a bad request.

This is the terraform code in question,

resource "confluent_custom_connector_plugin" "debezium_mongo" {
  # https://docs.confluent.io/cloud/current/connectors/bring-your-connector/custom-connector-qs.html#custom-connector-quick-start
  display_name                  = "Debezium MongoDB CDC Connector Plugin"
  documentation_link        = "https://debezium.io/documentation/reference/stable/connectors/mongodb.html"
  connector_class             = "io.debezium.connector.mongodb.MongoDbConnector"
  connector_type              = "SOURCE"
  cloud                               = "AZURE"
  sensitive_config_properties = []
  filename                          = "${path.module}/connectors/debezium-connector-mongodb-2.4.2.zip"
}

I'm not sure if this is an issue with the resource itself or if I'm missing something in my code.

Thanks for any help.

linouk23 commented 2 hours ago

@cconnorsada, thank you for creating the issue!

We’ve confirmed that this is a limitation of the TF Provider, and we have created an internal Jira ticket (APIT-2636) to address it.

As a quick workaround, could you try using the Confluent CLI to create a custom connector plugin and then use terraform import?

cconnorsada commented 2 hours ago

Hi @linouk23,

I will try with the import, it does limit our programmatic use of the Confluent Cloud platform though if we have a manual step in the cluster loop. Looking forward to a resolution in the future!

I'll let you know how it goes once I try it on Monday. Have a good weekend. :)