IBM-Cloud / terraform-provider-ibm

https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs
Mozilla Public License 2.0
342 stars 673 forks source link

ibm_container_ingress_instance not using private endpoint when using provider "visibility" "private #5657

Open fberzollaibm opened 2 months ago

fberzollaibm commented 2 months ago

Community Note

Terraform CLI and Terraform IBM Provider Version

Provider version : 1.67.1 Terraform Version : v1.6.6

Affected Resource(s)

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.


provider "ibm" {
  region     = var.region_target
  alias      = "target_account"
  visibility = "private"
}

resource "ibm_container_ingress_instance" "instance" {
  count = var.dry_run ? 0 : 1
  provider = ibm.target_account
  cluster         = local.cluster_name
  instance_crn    = data.ibm_resource_instance.secret_manager_instance[0].id
  secret_group_id = ibm_sm_secret_group.sm_secret_group-openshift[0].secret_group_id
  is_default      = true
  depends_on = [
    ibm_container_vpc_cluster.openshift-cluster,
    ibm_sm_secret_group.sm_secret_group-openshift,
    time_sleep.wait_120_seconds
  ]
}

Debug Output

2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.586Z [DEBUG] ibm_container_ingress_instance.instance[0]: applying the planned Create change 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.622Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: [DEBUG] REQUEST: [2024-09-19T13:46:54Z] POST /global/ingress/v2/secret/registerInstance HTTP/1.1 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.622Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Host: private.us-south.containers.cloud.ibm.com 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.622Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Accept: application/json 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Accept-Language: en 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Authorization: [PRIVATE DATA HIDDEN] 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Content-Type: application/json 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: User-Agent: Bluemix-go SDK 0.1 / linux 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Auth-Refresh-Token: [PRIVATE DATA HIDDEN] 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Original-User-Agent: terraform-provider-ibm/1.67.1 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.623Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: {"cluster":"carfr-npr-test-lro6","crn":"crn:v1:bluemix:public:secrets-manager:us-south:a/c73effe6a5be4c1e8a9a3b91e43b7165:4d4899a9-d0d8-4e6a-ab80-d689f6b8578f::","isDefault":true,"secretGroupID":"122d724b-a01e-988e-ac3d-5bbe74c07079"} 2024/09/19 13:46:54 Terraform apply | kubernetes_namespace.cert_utils_ns[0]: Modifying... [id=cert-utils-operator] 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.645Z [WARN] Provider "registry.terraform.io/hashicorp/kubernetes" produced an invalid plan for kubernetes_namespace.cert_utils_ns[0], but we are tolerating it because it is using the legacy plugin SDK. 2024/09/19 13:46:54 Terraform apply | The following problems may be the cause of any confusing errors from downstream operations: 2024/09/19 13:46:54 Terraform apply | - .wait_for_default_service_account: planned value cty.False for a non-computed attribute 2024/09/19 13:46:54 Terraform apply | - .metadata[0].generate_name: planned value cty.StringVal("") for a non-computed attribute 2024/09/19 13:46:54 Terraform apply | - .metadata[0].labels: planned value cty.MapValEmpty(cty.String) for a non-computed attribute 2024/09/19 13:46:54 Terraform apply | - .metadata[0].annotations: planned value cty.MapValEmpty(cty.String) for a non-computed attribute 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.645Z [INFO] Starting apply for kubernetes_namespace.cert_utils_ns[0] 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.646Z [DEBUG] kubernetes_namespace.cert_utils_ns[0]: applying the planned Update change 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.649Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: 2024/09/19 13:46:54 [INFO] Updating namespace: [{"path":"/metadata/annotations/openshift.io~1sa.scc.mcs","op":"remove"} {"path":"/metadata/annotations/openshift.io~1sa.scc.supplemental-groups","op":"remove"} {"path":"/metadata/annotations/openshift.io~1sa.scc.uid-range","op":"remove"}] 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.663Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: Sending HTTP Request: tf_http_req_uri=/api/v1/namespaces/cert-utils-operator Accept-Encoding=gzip Host=c130-e.private.us-south.containers.cloud.ibm.com:30182 User-Agent="HashiCorp/1.0 Terraform/1.6.6" tf_http_op_type=request @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.34.0/helper/logging/logging_http_transport.go:160 @module=kubernetes.Kubernetes Content-Length=239 Content-Type="application/json-patch+json" tf_http_req_method=PATCH tf_http_req_version=HTTP/1.1 tf_http_trans_id=6ae543e7-2538-0629-6cfa-99eef31c392d Accept="application/json, /" Authorization="Bearer sha256~QhPskfsKaQUHswSJdowhIV_proXKtAf-Py7P0he_Z3U" new_logger_warning="This log was generated by a subsystem logger that wasn't created before being used. Use tflog.NewSubsystem to create this logger before it is used." tf_http_req_body="[{\"path\":\"/metadata/annotations/openshift.io~1sa.scc.mcs\",\"op\":\"remove\"},{\"path\":\"/metadata/annotations/openshift.io~1sa.scc.supplemental-groups\",\"op\":\"remove\"},{\"path\":\"/metadata/annotations/openshift.io~1sa.scc.uid-range\",\"op\":\"remove\"}]" timestamp=2024-09-19T13:46:54.649Z 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.795Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: Received HTTP Response: @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.34.0/helper/logging/logging_http_transport.go:160 @module=kubernetes.Kubernetes Content-Type=application/json new_logger_warning="This log was generated by a subsystem logger that wasn't created before being used. Use tflog.NewSubsystem to create this logger before it is used." tf_http_res_status_code=200 tf_http_trans_id=6ae543e7-2538-0629-6cfa-99eef31c392d Cache-Control="no-cache, private" X-Kubernetes-Pf-Prioritylevel-Uid=52d9063c-43c9-4d80-9075-4b22746211a7 Audit-Id=8a6f254a-8177-4d11-afd3-d88bfae40c9d Content-Length=1115 Date="Thu, 19 Sep 2024 13:46:54 GMT" 2024/09/19 13:46:54 Terraform apply | tf_http_res_body= 2024/09/19 13:46:54 Terraform apply | | {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"cert-utils-operator","uid":"f4007b0f-27ed-4830-b785-ae41ac701699","resourceVersion":"111424","creationTimestamp":"2024-09-19T12:41:55Z","labels":{"kubernetes.io/metadata.name":"cert-utils-operator","pod-security.kubernetes.io/audit":"restricted","pod-security.kubernetes.io/audit-version":"v1.24","pod-security.kubernetes.io/warn":"restricted","pod-security.kubernetes.io/warn-version":"v1.24"},"managedFields":[{"manager":"pod-security-admission-label-synchronization-controller","operation":"Apply","apiVersion":"v1","time":"2024-09-19T12:41:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{"f:pod-security.kubernetes.io/audit":{},"f:pod-security.kubernetes.io/audit-version":{},"f:pod-security.kubernetes.io/warn":{},"f:pod-security.kubernetes.io/warn-version":{}}}}},{"manager":"HashiCorp","operation":"Update","apiVersion":"v1","time":"2024-09-19T12:41:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:kubernetes.io/metadata.name":{}}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2024/09/19 13:46:54 Terraform apply | Strict-Transport-Security=max-age=31536000 X-Kubernetes-Pf-Flowschema-Uid=5cd176fb-55f4-4730-b2a2-99075f21f927 tf_http_op_type=response tf_http_res_status_reason="200 OK" tf_http_res_version=HTTP/2.0 timestamp=2024-09-19T13:46:54.794Z 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.795Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: 2024/09/19 13:46:54 [INFO] Submitted updated namespace: &v1.Namespace{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cert-utils-operator", GenerateName:"", Namespace:"", SelfLink:"", UID:"f4007b0f-27ed-4830-b785-ae41ac701699", ResourceVersion:"111424", Generation:0, CreationTimestamp:time.Date(2024, time.September, 19, 12, 41, 55, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(int64)(nil), Labels:map[string]string{"kubernetes.io/metadata.name":"cert-utils-operator", "pod-security.kubernetes.io/audit":"restricted", "pod-security.kubernetes.io/audit-version":"v1.24", "pod-security.kubernetes.io/warn":"restricted", "pod-security.kubernetes.io/warn-version":"v1.24"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"pod-security-admission-label-synchronization-controller", Operation:"Apply", APIVersion:"v1", Time:time.Date(2024, time.September, 19, 12, 41, 55, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(v1.FieldsV1)(0xc0014e0d80), Subresource:""}, v1.ManagedFieldsEntry{Manager:"HashiCorp", Operation:"Update", APIVersion:"v1", Time:time.Date(2024, time.September, 19, 12, 41, 55, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(v1.FieldsV1)(0xc0014e0db0), Subresource:""}}}, Spec:v1.NamespaceSpec{Finalizers:[]v1.FinalizerName{"kubernetes"}}, Status:v1.NamespaceStatus{Phase:"Active", Conditions:[]v1.NamespaceCondition(nil)}} 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.796Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: 2024/09/19 13:46:54 [INFO] Checking namespace cert-utils-operator 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.796Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: Sending HTTP Request: User-Agent="HashiCorp/1.0 Terraform/1.6.6" tf_http_req_body="" tf_http_req_method=GET tf_http_trans_id=db8f3e22-c55a-f66e-a4ba-e1816f25a5d9 Accept-Encoding=gzip Authorization="Bearer sha256~QhPskfsKaQUHswSJdowhIV_proXKtAf-Py7P0he_Z3U" tf_http_req_uri=/api/v1/namespaces/cert-utils-operator tf_http_req_version=HTTP/1.1 @module=kubernetes.Kubernetes new_logger_warning="This log was generated by a subsystem logger that wasn't created before being used. Use tflog.NewSubsystem to create this logger before it is used." @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.34.0/helper/logging/logging_http_transport.go:160 Accept="application/json, /" tf_http_op_type=request Host=c130-e.private.us-south.containers.cloud.ibm.com:30182 timestamp=2024-09-19T13:46:54.796Z 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.813Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: Received HTTP Response: Cache-Control="no-cache, private" Date="Thu, 19 Sep 2024 13:46:54 GMT" tf_http_op_type=response tf_http_trans_id=db8f3e22-c55a-f66e-a4ba-e1816f25a5d9 @module=kubernetes.Kubernetes Content-Type=application/json X-Kubernetes-Pf-Flowschema-Uid=5cd176fb-55f4-4730-b2a2-99075f21f927 2024/09/19 13:46:54 Terraform apply | tf_http_res_body= 2024/09/19 13:46:54 Terraform apply | | {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"cert-utils-operator","uid":"f4007b0f-27ed-4830-b785-ae41ac701699","resourceVersion":"111424","creationTimestamp":"2024-09-19T12:41:55Z","labels":{"kubernetes.io/metadata.name":"cert-utils-operator","pod-security.kubernetes.io/audit":"restricted","pod-security.kubernetes.io/audit-version":"v1.24","pod-security.kubernetes.io/warn":"restricted","pod-security.kubernetes.io/warn-version":"v1.24"},"managedFields":[{"manager":"pod-security-admission-label-synchronization-controller","operation":"Apply","apiVersion":"v1","time":"2024-09-19T12:41:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{"f:pod-security.kubernetes.io/audit":{},"f:pod-security.kubernetes.io/audit-version":{},"f:pod-security.kubernetes.io/warn":{},"f:pod-security.kubernetes.io/warn-version":{}}}}},{"manager":"HashiCorp","operation":"Update","apiVersion":"v1","time":"2024-09-19T12:41:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:kubernetes.io/metadata.name":{}}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2024/09/19 13:46:54 Terraform apply | tf_http_res_status_reason="200 OK" @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.34.0/helper/logging/logging_http_transport.go:160 Audit-Id=d9db06b4-b932-41da-9a37-1c5b4d7777b0 Strict-Transport-Security=max-age=31536000 X-Kubernetes-Pf-Prioritylevel-Uid=52d9063c-43c9-4d80-9075-4b22746211a7 tf_http_res_version=HTTP/2.0 Content-Length=1115 tf_http_res_status_code=200 new_logger_warning="This log was generated by a subsystem logger that wasn't created before being used. Use tflog.NewSubsystem to create this logger before it is used." timestamp=2024-09-19T13:46:54.813Z 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.813Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: 2024/09/19 13:46:54 [INFO] Namespace cert-utils-operator exists 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.813Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: 2024/09/19 13:46:54 [INFO] Reading namespace cert-utils-operator 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.814Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: Sending HTTP Request: new_logger_warning="This log was generated by a subsystem logger that wasn't created before being used. Use tflog.NewSubsystem to create this logger before it is used." tf_http_req_body="" tf_http_trans_id=5784a635-6ce6-4203-8e8c-0ebb00887d2f tf_http_req_method=GET @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.34.0/helper/logging/logging_http_transport.go:160 @module=kubernetes.Kubernetes Accept-Encoding=gzip Authorization="Bearer sha256~QhPskfsKaQUHswSJdowhIV_proXKtAf-Py7P0he_Z3U" Host=c130-e.private.us-south.containers.cloud.ibm.com:30182 tf_http_req_version=HTTP/1.1 Accept="application/json, /" User-Agent="HashiCorp/1.0 Terraform/1.6.6" tf_http_op_type=request tf_http_req_uri=/api/v1/namespaces/cert-utils-operator timestamp=2024-09-19T13:46:54.814Z 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.830Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: Received HTTP Response: Content-Type=application/json tf_http_res_status_code=200 tf_http_res_version=HTTP/2.0 tf_http_trans_id=5784a635-6ce6-4203-8e8c-0ebb00887d2f Content-Length=1583 Date="Thu, 19 Sep 2024 13:46:54 GMT" Strict-Transport-Security=max-age=31536000 X-Kubernetes-Pf-Prioritylevel-Uid=52d9063c-43c9-4d80-9075-4b22746211a7 new_logger_warning="This log was generated by a subsystem logger that wasn't created before being used. Use tflog.NewSubsystem to create this logger before it is used." 2024/09/19 13:46:54 Terraform apply | tf_http_res_body= 2024/09/19 13:46:54 Terraform apply | | {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"cert-utils-operator","uid":"f4007b0f-27ed-4830-b785-ae41ac701699","resourceVersion":"111427","creationTimestamp":"2024-09-19T12:41:55Z","labels":{"kubernetes.io/metadata.name":"cert-utils-operator","pod-security.kubernetes.io/audit":"restricted","pod-security.kubernetes.io/audit-version":"v1.24","pod-security.kubernetes.io/warn":"restricted","pod-security.kubernetes.io/warn-version":"v1.24"},"annotations":{"openshift.io/sa.scc.mcs":"s0:c26,c10","openshift.io/sa.scc.supplemental-groups":"1000670000/10000","openshift.io/sa.scc.uid-range":"1000670000/10000"},"managedFields":[{"manager":"pod-security-admission-label-synchronization-controller","operation":"Apply","apiVersion":"v1","time":"2024-09-19T12:41:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{"f:pod-security.kubernetes.io/audit":{},"f:pod-security.kubernetes.io/audit-version":{},"f:pod-security.kubernetes.io/warn":{},"f:pod-security.kubernetes.io/warn-version":{}}}}},{"manager":"HashiCorp","operation":"Update","apiVersion":"v1","time":"2024-09-19T12:41:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:kubernetes.io/metadata.name":{}}}}},{"manager":"cluster-policy-controller","operation":"Update","apiVersion":"v1","time":"2024-09-19T13:46:54Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:openshift.io/sa.scc.mcs":{},"f:openshift.io/sa.scc.supplemental-groups":{},"f:openshift.io/sa.scc.uid-range":{}}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2024/09/19 13:46:54 Terraform apply | @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.34.0/helper/logging/logging_http_transport.go:160 Audit-Id=3c25dcdf-d799-4891-9c91-b3552b650bb5 Cache-Control="no-cache, private" X-Kubernetes-Pf-Flowschema-Uid=5cd176fb-55f4-4730-b2a2-99075f21f927 @module=kubernetes.Kubernetes tf_http_op_type=response tf_http_res_status_reason="200 OK" timestamp=2024-09-19T13:46:54.829Z 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.830Z [DEBUG] provider.terraform-provider-kubernetes_v2.32.0_x5: 2024/09/19 13:46:54 [INFO] Received namespace: &v1.Namespace{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cert-utils-operator", GenerateName:"", Namespace:"", SelfLink:"", UID:"f4007b0f-27ed-4830-b785-ae41ac701699", ResourceVersion:"111427", Generation:0, CreationTimestamp:time.Date(2024, time.September, 19, 12, 41, 55, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(int64)(nil), Labels:map[string]string{"kubernetes.io/metadata.name":"cert-utils-operator", "pod-security.kubernetes.io/audit":"restricted", "pod-security.kubernetes.io/audit-version":"v1.24", "pod-security.kubernetes.io/warn":"restricted", "pod-security.kubernetes.io/warn-version":"v1.24"}, Annotations:map[string]string{"openshift.io/sa.scc.mcs":"s0:c26,c10", "openshift.io/sa.scc.supplemental-groups":"1000670000/10000", "openshift.io/sa.scc.uid-range":"1000670000/10000"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"pod-security-admission-label-synchronization-controller", Operation:"Apply", APIVersion:"v1", Time:time.Date(2024, time.September, 19, 12, 41, 55, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(v1.FieldsV1)(0xc0007cec18), Subresource:""}, v1.ManagedFieldsEntry{Manager:"HashiCorp", Operation:"Update", APIVersion:"v1", Time:time.Date(2024, time.September, 19, 12, 41, 55, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(v1.FieldsV1)(0xc0007cec48), Subresource:""}, v1.ManagedFieldsEntry{Manager:"cluster-policy-controller", Operation:"Update", APIVersion:"v1", Time:time.Date(2024, time.September, 19, 13, 46, 54, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(v1.FieldsV1)(0xc0007cec78), Subresource:""}}}, Spec:v1.NamespaceSpec{Finalizers:[]v1.FinalizerName{"kubernetes"}}, Status:v1.NamespaceStatus{Phase:"Active", Conditions:[]v1.NamespaceCondition(nil)}} 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.832Z [WARN] Provider "provider[\"registry.terraform.io/hashicorp/kubernetes\"]" produced an unexpected new value for kubernetes_namespace.cert_utils_ns[0], but we are tolerating it because it is using the legacy plugin SDK. 2024/09/19 13:46:54 Terraform apply | The following problems may be the cause of any confusing errors from downstream operations: 2024/09/19 13:46:54 Terraform apply | - .metadata[0].annotations: new element "openshift.io/sa.scc.mcs" has appeared 2024/09/19 13:46:54 Terraform apply | - .metadata[0].annotations: new element "openshift.io/sa.scc.supplemental-groups" has appeared 2024/09/19 13:46:54 Terraform apply | - .metadata[0].annotations: new element "openshift.io/sa.scc.uid-range" has appeared 2024/09/19 13:46:54 Terraform apply | - .metadata[0].resource_version: was cty.StringVal("94243"), but now cty.StringVal("111427") 2024/09/19 13:46:54 Terraform apply | kubernetes_namespace.cert_utils_ns[0]: Modifications complete after 0s [id=cert-utils-operator] 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.868Z [DEBUG] State storage statemgr.Filesystem declined to persist a state snapshot 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.869Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF" 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.882Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/kubernetes/2.32.0/linux_amd64/terraform-provider-kubernetes_v2.32.0_x5 pid=286 2024/09/19 13:46:54 Terraform apply | 2024-09-19T13:46:54.883Z [DEBUG] provider: plugin exited 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: [DEBUG] RESPONSE: [2024-09-19T13:46:58Z] Elapsed: 3825ms HTTP/1.1 400 Bad Request 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Content-Length: 359 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Cache-Control: no-cache, no-store 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Content-Security-Policy: default-src 'none'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; connect-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-inline'; font-src 'self' 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Content-Type: application/json; charset=utf-8 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Date: Thu, 19 Sep 2024 13:46:58 GMT 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Expires: 0 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Pragma: no-cache 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: Strict-Transport-Security: max-age=31536000; includeSubDomains 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Carrier: prod-dal10-carrier105 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Content-Type-Options: nosniff 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Correlation-Id: 3d4906f1-d430-403b-b700-bb4508ec19ff 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Frame-Options: DENY 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Hostname: armada-global-api-d6654ffdd-rr9mz 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Permitted-Cross-Domain-Policies: master-only 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Region: us-south 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Request-Id: df1e504b-89f9-4752-84b1-d3ec299ea582 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: X-Xss-Protection: 1; mode=block 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: {"incidentID":"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582","code":"ECICGCA","description":"Unable to fetch resource service instance.","type":"General","recoveryCLI":"Verify that the CRN is correct and that you have permission to access your instance. Ensure your instance is listed under ibmcloud resource service-instances"} 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.430Z [DEBUG] provider.terraform-provider-ibm_v1.67.1: --- 2024/09/19 13:46:58 Terraform apply | id: terraform-00199cf5 2024/09/19 13:46:58 Terraform apply | summary: 'Request failed with status code: 400, ServerErrorResponse: {"incidentID":"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582","code":"ECICGCA","description":"Unable 2024/09/19 13:46:58 Terraform apply | to fetch resource service instance.","type":"General","recoveryCLI":"Verify that 2024/09/19 13:46:58 Terraform apply | the CRN is correct and that you have permission to access your instance. Ensure 2024/09/19 13:46:58 Terraform apply | your instance is listed under ibmcloud resource service-instances"}' 2024/09/19 13:46:58 Terraform apply | severity: error 2024/09/19 13:46:58 Terraform apply | resource: ibm_container_ingress_instance 2024/09/19 13:46:58 Terraform apply | operation: create 2024/09/19 13:46:58 Terraform apply | component: 2024/09/19 13:46:58 Terraform apply | name: github.com/IBM-Cloud/terraform-provider-ibm 2024/09/19 13:46:58 Terraform apply | version: 1.67.1 2024/09/19 13:46:58 Terraform apply | ---: timestamp=2024-09-19T13:46:58.430Z 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.431Z [ERROR] provider.terraform-provider-ibm_v1.67.1: Response contains error diagnostic: tf_proto_version=5.4 tf_resource_type=ibm_container_ingress_instance 2024/09/19 13:46:58 Terraform apply | diagnostic_detail= 2024/09/19 13:46:58 Terraform apply | | --- 2024/09/19 13:46:58 Terraform apply | | id: terraform-00199cf5 2024/09/19 13:46:58 Terraform apply | | summary: 'Request failed with status code: 400, ServerErrorResponse: {"incidentID":"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582","code":"ECICGCA","description":"Unable 2024/09/19 13:46:58 Terraform apply | | to fetch resource service instance.","type":"General","recoveryCLI":"Verify that 2024/09/19 13:46:58 Terraform apply | | the CRN is correct and that you have permission to access your instance. Ensure 2024/09/19 13:46:58 Terraform apply | | your instance is listed under ibmcloud resource service-instances"}' 2024/09/19 13:46:58 Terraform apply | | severity: error 2024/09/19 13:46:58 Terraform apply | | resource: ibm_container_ingress_instance 2024/09/19 13:46:58 Terraform apply | | operation: create 2024/09/19 13:46:58 Terraform apply | | component: 2024/09/19 13:46:58 Terraform apply | | name: github.com/IBM-Cloud/terraform-provider-ibm 2024/09/19 13:46:58 Terraform apply | | version: 1.67.1 2024/09/19 13:46:58 Terraform apply | | --- 2024/09/19 13:46:58 Terraform apply | @caller=github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/diag/diagnostics.go:58 diagnostic_summary="Request failed with status code: 400, ServerErrorResponse: {\"incidentID\":\"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582\",\"code\":\"ECICGCA\",\"description\":\"Unable to fetch resource service instance.\",\"type\":\"General\",\"recoveryCLI\":\"Verify that the CRN is correct and that you have permission to access your instance. Ensure your instance is listed under ibmcloud resource service-instances\"}" tf_provider_addr=provider tf_rpc=ApplyResourceChange @module=sdk.proto diagnostic_severity=ERROR tf_req_id=46416262-2ca8-b688-f5f7-aaa6dc856088 timestamp=2024-09-19T13:46:58.430Z 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.468Z [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot 2024/09/19 13:46:58 Terraform apply | 2024-09-19T13:46:58.468Z [ERROR] vertex "ibm_container_ingress_instance.instance[0]" error: Request failed with status code: 400, ServerErrorResponse: {"incidentID":"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582","code":"ECICGCA","description":"Unable to fetch resource service instance.","type":"General","recoveryCLI":"Verify that the CRN is correct and that you have permission to access your instance. Ensure your instance is listed under ibmcloud resource service-instances"} 2024/09/19 13:46:58 Terraform apply | 2024/09/19 13:46:58 Terraform apply | Error: Request failed with status code: 400, ServerErrorResponse: {"incidentID":"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582","code":"ECICGCA","description":"Unable to fetch resource service instance.","type":"General","recoveryCLI":"Verify that the CRN is correct and that you have permission to access your instance. Ensure your instance is listed under ibmcloud resource service-instances"} 2024/09/19 13:46:58 Terraform apply | 2024/09/19 13:46:58 Terraform apply | with ibm_container_ingress_instance.instance[0], 2024/09/19 13:46:58 Terraform apply | on openshift.tf line 68, in resource "ibm_container_ingress_instance" "instance": 2024/09/19 13:46:58 Terraform apply | 68: resource "ibm_container_ingress_instance" "instance" { 2024/09/19 13:46:58 Terraform apply | 2024/09/19 13:46:58 Terraform apply | --- 2024/09/19 13:46:58 Terraform apply | id: terraform-00199cf5 2024/09/19 13:46:58 Terraform apply | summary: 'Request failed with status code: 400, ServerErrorResponse: 2024/09/19 13:46:58 Terraform apply | {"incidentID":"df1e504b-89f9-a752-84b1-d3ec299ea582,df1e504b-89f9-a752-84b1-d3ec299ea582","code":"ECICGCA","description":"Unable 2024/09/19 13:46:58 Terraform apply | to fetch resource service instance.","type":"General","recoveryCLI":"Verify that 2024/09/19 13:46:58 Terraform apply | the CRN is correct and that you have permission to access your instance. Ensure 2024/09/19 13:46:58 Terraform apply | your instance is listed under ibmcloud resource service-instances"}' 2024/09/19 13:46:58 Terraform apply | severity: error 2024/09/19 13:46:58 Terraform apply | resource: ibm_container_ingress_instance 2024/09/19 13:46:58 Terraform apply | operation: create 2024/09/19 13:46:58 Terraform apply | component: 2024/09/19 13:46:58 Terraform apply | name: github.com/IBM-Cloud/terraform-provider-ibm 2024/09/19 13:46:58 Terraform apply | version: 1.67.1 2024/09/19 13:46:58 Terraform apply | ---

Expected Behavior

We have activated CBR rules to use private endpoints for All services (secret manager....)

The IBM CLoud provider configuration specifies that we want to use private endpoints visibility = "private"

The provider must use private endpoints to access IBM Cloud APIS

Actual Behavior

When trying to create a ibm_container_ingress_instance we pass a reference to the secret manager crn.

When CBR rules are activated the ibm_container_ingress_instance failed with HTTP 400 Error code

When CBR rules a deactivated no error.

Steps to Reproduce

hkantare commented 2 months ago

From the logs we see Host: private.us-south.containers.cloud.ibm.com for ibmcloud Host=c130-e.private.us-south.containers.cloud.ibm.com:30182 for kubernetes We see it uses private endpoint only

hkantare commented 2 months ago

Can you upload the complete log

fberzollaibm commented 2 months ago

Full Log [Uploading all-debug-log.txt…]()

fberzollaibm commented 2 months ago

CBR Activity Tracker deny cbr-deny-ecrets-manager-instance-retrieve.json cbr-deny-secrets-manager--endpoints-view.json

fberzollaibm commented 2 months ago

Maybe the issue is on the API side : https://cloud.ibm.com/apidocs/kubernetes/containers-v1-v2#registersecretinstance