Unable to create `ibm_container_vpc_cluster` resources using a non-default `Resource Group` and `Service ID API Key`

lantins commented 1 week ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

$ terraform -v
Terraform v1.9.5
on darwin_amd64
+ provider registry.terraform.io/ibm-cloud/ibm v1.68.1

Affected Resource(s)

ibm_container_vpc_cluster

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# ------------------------------------------------------------------------------

terraform {
  required_version = "~> 1.9"

  required_providers {
    ibm = {
      source  = "IBM-Cloud/ibm"
      version = "1.68.1"
    }
  }
}

provider "ibm" {
  region = "us-east"
  # ibmcloud_api_key - set by environment variable `IBMCLOUD_API_KEY`
}

# --- Resource Group -----------------------------------------------------------

resource "ibm_resource_group" "env" {
  name = "chronicle - stage"
  tags = ["env:stage", "product:chronicle"]
}

# --- VPC & Networking ---------------------------------------------------------

resource "ibm_is_vpc" "env" {
  name           = "chronicle-stage"
  resource_group = ibm_resource_group.env.id
  classic_access = false
}

resource "ibm_is_public_gateway" "gw-1" {
  name           = "public-gateway-1"
  vpc            = ibm_is_vpc.env.id
  zone           = "us-east-1"
  resource_group = ibm_resource_group.env.id
}

resource "ibm_is_subnet" "us-east-1" {
  name                     = "chronicle-stage-us-east-1"
  vpc                      = ibm_is_vpc.env.id
  zone                     = "us-east-1"
  total_ipv4_address_count = 1024
  public_gateway           = ibm_is_public_gateway.gw-1.id
  resource_group           = ibm_resource_group.env.id
}

resource "ibm_is_public_gateway" "gw-2" {
  name           = "public-gateway-2"
  vpc            = ibm_is_vpc.env.id
  zone           = "us-east-2"
  resource_group = ibm_resource_group.env.id
}

resource "ibm_is_subnet" "us-east-2" {
  name                     = "chronicle-stage-us-east-2"
  vpc                      = ibm_is_vpc.env.id
  zone                     = "us-east-2"
  total_ipv4_address_count = 1024
  public_gateway           = ibm_is_public_gateway.gw-2.id
  resource_group           = ibm_resource_group.env.id
}

# --- K8S Cluster --------------------------------------------------------------

resource "ibm_container_vpc_cluster" "chronicle" {
  name              = "chronicle-stage"
  vpc_id            = ibm_is_vpc.env.id
  kube_version      = "1.30.4"
  resource_group_id = ibm_resource_group.env.id

  # default worker pool
  flavor       = "bx2.2x8"
  worker_count = "4"

  disable_public_service_endpoint = true

  zones {
    subnet_id = ibm_is_subnet.us-east-1.id
    name      = "us-east-1"
  }
  zones {
    subnet_id = ibm_is_subnet.us-east-2.id
    name      = "us-east-2"
  }
}

Debug Output

https://gist.github.com/lantins/584b988cf401de3be6ca60e502d648a2

Panic Output

n/a

Expected Behavior

The IKS cluster should be created without error.

Actual Behavior

IKS cluster fails to be created, this seems related to using a non-default Resource Group (i.e. setting resource_group_id). The error received is in relation to it trying to create a containers-kubernetes-key IAM User API Key and fails because I've used a Service ID API Key.

Steps to Reproduce

Create a Service ID and create a API Key associated with it.
Set the environment variable IBMCLOUD_API_KEY to use the Service ID API Key.
terraform apply

Important Factoids

When creating a IKS cluster using the ibmcloud CLI using a Service ID API Key it is able to create the cluster without any issues. It does not result in a containers-kubernetes-key or similar being created/saved anywhere as far as I can tell.

References

0000

lantins commented 1 week ago

Some extra context, I've just tried the ibmcloud CLI with a Resource Group and it has the same error/issue.

I figured I could always call ibmcloud using a null_resource as a workaround for a short time, but it seems that isn't the case since I need the cluster to be part of a non-default Resource Group.

FAILS:

$ ibmcloud target -g "chronicle - stage"
$ ibmcloud ks cluster create vpc-gen2 --name created-with-service-id --zone us-east-1 --vpc-id r014-d69c0c97-e542-451b-84db-864d9ae32386 --subnet-id 0757-eddfd4cb-8643-4ffd-87b4-1901f6d6a389 --flavor bx2.2x8
Creating cluster...
FAILED
Failed to create an API key with IAM. Revise your request and try again. (A03e9b)

Incident ID: e8ccc832-bb53-4cc1-8293-0ee1260f20e5

WORKS:

$ ibmcloud target -g ""
$ ibmcloud ks cluster create vpc-gen2 --name created-with-service-id --zone us-east-1 --vpc-id r014-d69c0c97-e542-451b-84db-864d9ae32386 --subnet-id 0757-eddfd4cb-8643-4ffd-87b4-1901f6d6a389 --flavor bx2.2x8
Creating cluster...
OK
Cluster created with ID crcdlr8w0oetmlrvp7j0

lantins commented 1 week ago

After a bit more digging, I think the only difference in requests to the API is if the X-Auth-Resource-Group header is set.

So to me, this seems like its a IBM Cloud API 'backend' issue? rather than an issue with the Terraform Provider?

IBM-Cloud / terraform-provider-ibm