Using data "confluent_schema_registry_cluster" with aliased provider failing

[lmeadors]

I am trying to convert the example at examples/configurations/single-event-types-proto-schema-with-alias to use the new approach to defining environments and schema registry, but it fails for me.

I changed the environment and schema registry resources to look like this:

resource "confluent_environment" "staging" {
  display_name = "Staging"
  stream_governance {
    package = "ESSENTIALS"
  }
}

data "confluent_schema_registry_cluster" "essentials" {
  environment {
    id = confluent_environment.staging.id
  }
}

Then I changed the provider to look like this:

provider "confluent" {
  # https://developer.hashicorp.com/terraform/language/providers/configuration#alias-multiple-provider-configurations
  alias = "schema-registry"

  schema_registry_id            = data.confluent_schema_registry_cluster.essentials.id
  schema_registry_rest_endpoint = data.confluent_schema_registry_cluster.essentials.rest_endpoint
  schema_registry_api_key       = confluent_api_key.env-manager-schema-registry-api-key.id
  schema_registry_api_secret    = confluent_api_key.env-manager-schema-registry-api-key.secret
}

Running terraform apply fails the first time with this error:

│ Error: error reading Schema Registry Clusters: there are no SR clusters in "env-xxxxxx" environment
│
│   with data.confluent_schema_registry_cluster.essentials,
│   on main.tf line 52, in data "confluent_schema_registry_cluster" "essentials":
│   52: data "confluent_schema_registry_cluster" "essentials" {

Running it a second time, it fails with this:

│ Error: All 4 schema_registry_api_key, schema_registry_api_secret, schema_registry_rest_endpoint, schema_registry_id attributes should be set or not set in the provider block at the same time
│
│   with provider["registry.terraform.io/confluentinc/confluent"].schema-registry,
│   on main.tf line 19, in provider "confluent":
│   19: provider "confluent" {

If I run this: terraform apply -target=confluent_api_key.env-manager-schema-registry-api-key then things get better - that creates the API key and then I can run terraform apply to create everything else.

[linouk23]

@lmeadors thanks for creating this issue!

I suspect it's a general terraform issue where it requires all provider inputs to be known beforehand (i.e., passed as module variables or hardcoded), and in your scenario, confluent_api_key.env-manager-schema-registry-api-key.id = "". I'd recommend refactoring your code to use a module instead, to ensure these attributes aren't empty.

[lmeadors]

The provider not having the api key is somewhat incidental.

The bigger issue (in my mind, anyway) is that current example fails on the first application of it. I think that means that the environment resource is reporting that is all done and created before it actually is.

That in turn makes the data block fail to find the schema registry created by the environment resource falsely reporting that there are no SR clusters in "env-xxxxxx" environment, and the recovery from that failure is unintuitive.

It would be great if the environment returned the schema registry id (if defined) or that the data block to get the SR tried harder to find it. :D

I am not disagreeing that a more modular approach would be better (and I ended up refactoring my code to be that way), but rather that the example should either work as is or demonstrate the best practice.

[linouk23]

the example should either work as is

💯

@lmeadors could you share the example URL so we could update it?

[lmeadors]

I started with this:

https://github.com/confluentinc/terraform-provider-confluent/tree/master/examples/configurations/single-event-types-proto-schema-with-alias

Then applied the changes mentioned here:

https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/guides/version-2-upgrade

I would be happy to send a PR later this week if needed.