Issue with ibm_container_cluster_config / ROKS

[pauljegouic]

Hello there,

Following @hkantare demand, here is the bug regarding ibm_container_cluster_config.

Hypothesis:

• Use a ServiceID to execute the provisioning of a ROKS cluster.

Current behaviour

• Works locally,
• Does not work in Schematics.

Logs:

➜  ~ ic sch logs -i eu-de.workspace.globalcatalog-collection.70c905b8   
 2023/05/26 08:57:55 -----  New Workspace Action  -----
 2023/05/26 08:57:55 Request: activitId=ce64da3205cb52debbbb0e2cf37b41c5, account=3f2cae45b0644f6d87aefcf404f5987f, owner=ServiceId-569327e8-6a68-40d2-a32e-d361da1f48ca, requestID=f480f013-0de1-4e37-963d-ab098391e353
 2023/05/26 08:57:56 Related Activity: action=APPLY, workspaceID=eu-de.workspace.globalcatalog-collection.70c905b8, processedBy=orchestrator-79b9fcf969-w9b7b
 2023/05/26 08:57:56 Related Workspace: name=test-pje-eu-de, sourcerelease=(not specified), sourceurl=[TRUNCATED], folder=openshift-namespace-mngt-addon-2.0.0
 2023/05/26 08:57:59  --- Ready to execute the command --- 
 2023/05/26 08:58:00 workspace.template.StateFile: 9a7c8ff2-c3c7-4c96-a057-9dacb1e75e07
 2023/05/26 08:58:02 workspace.template.EnvFile: e6ffed5c-eb7d-43c8-b12d-16e83fe2abe5
 2023/05/26 08:58:10 workspace.template.SecFile: 32da49ba-7123-4a58-8f47-c6abeee01534
 2023/05/26 08:57:59 -----  New Action  -----
 2023/05/26 08:57:59 Request: requestID=f480f013-0de1-4e37-963d-ab098391e353
 2023/05/26 08:58:12 Related Activity: action=Apply, workspaceID=eu-de.workspace.globalcatalog-collection.70c905b8, processedByOrchestrator=f480f013-0de1-4e37-963d-ab098391e353_ce64da3205cb52debbbb0e2cf37b41c5, processedByJob=job12-6b57b96459-j8nmd, actionType=Terraform

 2023/05/26 08:58:17 -----  Terraform INIT  -----

 2023/05/26 08:58:17 Starting command: terraform1.3 init -input=false -no-color
 2023/05/26 08:58:17 Starting command: terraform1.3 init -input=false -no-color
 2023/05/26 08:58:19 Terraform init | 
 2023/05/26 08:58:19 Terraform init | Initializing the backend...
 2023/05/26 08:58:19 Terraform init | 
 2023/05/26 08:58:19 Terraform init | Initializing provider plugins...
 2023/05/26 08:58:19 Terraform init | - Finding latest version of hashicorp/null...
 2023/05/26 08:58:20 Terraform init | - Finding ibm-cloud/ibm versions matching "~> 1.26"...
 2023/05/26 08:58:20 Terraform init | - Finding hashicorp/kubernetes versions matching "~> 2.0"...
 2023/05/26 08:58:20 Terraform init | - Installing hashicorp/null v3.2.1...
 2023/05/26 08:58:21 Terraform init | - Installed hashicorp/null v3.2.1 (signed by HashiCorp)
 2023/05/26 08:58:22 Terraform init | - Installing ibm-cloud/ibm v1.53.0...
 2023/05/26 08:58:29 Terraform init | - Installed ibm-cloud/ibm v1.53.0 (self-signed, key ID AAD3B791C49CC253)
 2023/05/26 08:58:29 Terraform init | - Installing hashicorp/kubernetes v2.20.0...
 2023/05/26 08:58:33 Terraform init | - Installed hashicorp/kubernetes v2.20.0 (signed by HashiCorp)
 2023/05/26 08:58:33 Terraform init | 
 2023/05/26 08:58:33 Terraform init | Partner and community providers are signed by their developers.
 2023/05/26 08:58:33 Terraform init | If you'd like to know more about provider signing, you can read about it here:
 2023/05/26 08:58:33 Terraform init | https://www.terraform.io/docs/cli/plugins/signing.html
 2023/05/26 08:58:33 Terraform init | 
 2023/05/26 08:58:33 Terraform init | Terraform has created a lock file .terraform.lock.hcl to record the provider
 2023/05/26 08:58:33 Terraform init | selections it made above. Include this file in your version control repository
 2023/05/26 08:58:33 Terraform init | so that Terraform can guarantee to make the same selections by default when
 2023/05/26 08:58:33 Terraform init | you run "terraform init" in the future.
 2023/05/26 08:58:33 Terraform init | 
 2023/05/26 08:58:33 Terraform init | Terraform has been successfully initialized!
 2023/05/26 08:58:33 Command finished successfully.

 2023/05/26 08:58:33 -----  Terraform APPLY  -----

 2023/05/26 08:58:33 Starting command: terraform1.3 apply -state=terraform.tfstate -var-file=schematics.tfvars -auto-approve -no-color
 2023/05/26 08:58:33 Starting command: terraform1.3 apply -state=terraform.tfstate -var-file=schematics.tfvars -auto-approve -no-color
 2023/05/26 08:58:53 Terraform apply | data.ibm_resource_group.resource_group: Reading...
 2023/05/26 08:58:54 Terraform apply | data.ibm_resource_group.resource_group: Read complete after 1s [id=ba11aa2abb6b494480840577b92fd946]
 2023/05/26 08:58:54 Terraform apply | data.ibm_container_cluster_config.cluster_config: Reading...
 2023/05/26 08:59:01 Terraform apply | 
 2023/05/26 08:59:01 Terraform apply | Error: [ERROR] Error downloading the cluster config [opspf-cluster]: Request failed with status code: 400, BXNIM0453E: The refresh token contains subject type 'ServiceId', which is not valid for the intended operation. Supported subject types are UserId, Profile.
 2023/05/26 08:59:01 Terraform apply | 
 2023/05/26 08:59:01 Terraform apply |   with data.ibm_container_cluster_config.cluster_config,
 2023/05/26 08:59:01 Terraform apply |   on datasources.tf line 7, in data "ibm_container_cluster_config" "cluster_config":
 2023/05/26 08:59:01 Terraform apply |    7: data "ibm_container_cluster_config" "cluster_config" {
 2023/05/26 08:59:01 Terraform apply | 
 2023/05/26 08:59:01 Terraform APPLY error: Terraform APPLY errorexit status 1
 2023/05/26 08:59:01 Could not execute job: Error : Terraform APPLY errorexit status 1

OK

Works fine with an IKS, but lead to error with ROKS when executed over Schematics.

[hkantare]

@attilatabori /Lewis Evan Can you look into this issue @pauljegouic In local you pointed it works fine . In local are you using serviceID or user APIkey

[pauljegouic]

Service ID !

[pauljegouic]

Any updates ?

[hkantare]

@pauljegouic Can you share sample configuration files and the service ID (policies defined for services)

[pradeep-b]

From Paul JEGOUIC: Just to remind: With ServiceID API Key - Manager/Administrator on IKS/ROKS Service: I can pull the cluster config from CLI, I can pull the cluster config from terraform locally using the terraform resource ibm_cluster_config (sorry if not exact) (ROKS or IKS) I can pull the cluster config from terraform on Schematics - if cluster_type == IKS I cannot pull the cluster config from terraform on Schematics, if cluster_type == ROKS Latest usecase ends up with this message:

Error: [ERROR] Error downloading the cluster config [opspf-cluster]: Request failed with status code: 400, BXNIM0453E: The refresh token contains subject type 'ServiceId', which is not valid for the intended operation. Supported subject types are UserId, Profile.