Provisioning of Cloud logs instance fails when cos buckets are attached in parameters

maheshwarishikha commented 2 weeks ago

Provisioning of ICL fails if COS bucket details are provided in parameters, though all required auth policies are in-place.

 Error: [ERROR] Error waiting for create resource instance (crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::) to be succeeded: [ERROR] The resource instance 'crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::' creation failed: <nil>
│ 
│   with ibm_resource_instance.cloud_logs,
│   on test.tf line 62, in resource "ibm_resource_instance" "cloud_logs":
│   62: resource "ibm_resource_instance" "cloud_logs" {
│ 
│ ---
│ id: terraform-446cc00a
│ summary: '[ERROR] Error waiting for create resource instance (crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::)
│   to be succeeded: [ERROR] The resource instance ''crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::''
│   creation failed: <nil>'
│ severity: error
│ resource: ibm_resource_instance
│ operation: create
│ component:
│   name: github.com/IBM-Cloud/terraform-provider-ibm
│   version: 1.71.0

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

tf 1.9.2 provider 1.70.0 / 1.71.1

Affected Resource(s)

ibm_resource_instance

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please share a link to the ZIP file.
provider "ibm" {
  ibmcloud_api_key = var.ibmcloud_api_key
  region           = "us-south"
}

terraform {
  required_version = ">= 1.3.0"
  required_providers {
    ibm = {
      source  = "ibm-cloud/ibm"
      version = "1.71.0"
    }
    time = {
      source  = "hashicorp/time"
      version = "0.12.1"
    }
  }
}

variable "ibmcloud_api_key" {
  type        = string
  description = "The IBM Cloud API Token"
  sensitive   = true
}

data "ibm_resource_group" "cos_group" {
  name = "Default"
}

resource "ibm_resource_instance" "cos_instance" {
  name              = "debug-cos"
  resource_group_id = data.ibm_resource_group.cos_group.id
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
  tags              = []
}

resource "ibm_cos_bucket" "cos_bucket" {
  bucket_name          = "debug-icl-bucket"
  resource_instance_id = ibm_resource_instance.cos_instance.id
  storage_class        = "standard"
  endpoint_type = "public"
  region_location = "us-south"
}

resource "ibm_iam_authorization_policy" "cos_policy" {
  source_service_name      = "logs"
  source_resource_group_id = data.ibm_resource_group.cos_group.id
  roles                    = ["Writer"]
  description              = "Allow Cloud logs instances `Writer` access to the COS bucket."
  target_service_name         = "cloud-object-storage"
  target_resource_instance_id = ibm_resource_instance.cos_instance.id
}

resource "time_sleep" "wait_for_cos_authorization_policy" {
  depends_on = [ibm_iam_authorization_policy.cos_policy]
  create_duration = "30s"
}

resource "ibm_resource_instance" "cloud_logs" {
  depends_on        = [time_sleep.wait_for_cos_authorization_policy]
  name              = "debug-cloudlogs"
  resource_group_id = data.ibm_resource_group.cos_group.id
  service           = "logs"
  plan              = "standard"
  tags              = []
  location          = "us-south"
  parameters = {
    "logs_bucket_crn"         = ibm_cos_bucket.cos_bucket.crn
    "logs_bucket_endpoint"    = ibm_cos_bucket.cos_bucket.s3_endpoint_public
  }
}

Debug Output

Panic Output

Expected Behavior

It should create Cloud logs instance successfully.

Actual Behavior

Provisioning fails.

Steps to Reproduce

terraform apply

Important Factoids

References

0000

kavya498 commented 2 weeks ago

@maheshwarishikha, Looks like the authorisation policy is not set right. It is not been used.. Can you use guid instead of id for target_instance_id? target_resource_instance_id = ibm_resource_instance.cos_instance.id ==> target_resource_instance_id = ibm_resource_instance.cos_instance.guid something like below?

resource "ibm_iam_authorization_policy" "cos_policy" {
  source_service_name      = "logs"
  source_resource_group_id = data.ibm_resource_group.cos_group.id
  roles                    = ["Writer"]
  description              = "Allow Cloud logs instances `Writer` access to the COS bucket."
  target_service_name         = "cloud-object-storage"
  target_resource_instance_id = ibm_resource_instance.cos_instance.guid
}

maheshwarishikha commented 2 weeks ago

Thanks for the response @kavya498

Actually that was typo when I created a test code, changing it to guid worked for me. But we have auth policy in GoldenEye code scoped to the bucket like this https://github.com/terraform-ibm-modules/terraform-ibm-observability-instances/blob/main/modules/cloud_logs/main.tf#L44 that was working all fine few days back and now giving error. Checking more on this...

maheshwarishikha commented 2 weeks ago

@kavya498 , If we changed our auth policy like as below....i.e. scoped to the cos bucket

resource "ibm_iam_authorization_policy" "cos_policy" {
  source_service_name      = "logs"
  source_resource_group_id = data.ibm_resource_group.cos_group.id
  roles                    = ["Writer"]
  description              = "Allow Cloud logs instances `Writer` access to the COS bucket."
#   target_service_name         = "cloud-object-storage"
#   target_resource_instance_id = ibm_resource_instance.cos_instance.guid

  resource_attributes {
    name     = "serviceName"
    operator = "stringEquals"
    value    = "cloud-object-storage"
  }

  resource_attributes {
    name     = "accountId"
    operator = "stringEquals"
    value    = "xxx"
  }

  resource_attributes {
    name     = "serviceInstance"
    operator = "stringEquals"
    value    = ibm_resource_instance.cos_instance.guid
  }

  resource_attributes {
    name     = "resourceType"
    operator = "stringEquals"
    value    = "bucket"
  }

  resource_attributes {
    name     = "resource"
    operator = "stringEquals"
    value    = "debug-icl-bucket"
  }
}

ICL provisioning fails. It was working all fine few days back. Can you please help to find out whats missing in auth policy?? Or it is happening because of any change in provider code.

maheshwarishikha commented 2 weeks ago

Nopes, recreated everything and it worked this time. Checking in GoldenEye Module now... .

ocofaigh commented 2 weeks ago

@kavya498 It would be nice if the service could respond with a nicer error message if you know that its an issue related to auth policy. Is that something you plan on improving?

maheshwarishikha commented 4 days ago

Issue identified is with the metrics bucket attachment. Cloud logs team is working on this currently - internal issue to track is https://github.ibm.com/Observability/logs-support/issues/223

Regarding the proper error message, it is being tracked by https://github.com/IBM-Cloud/terraform-provider-ibm/issues/5729

Regarding the fix related to metrics bucket attachment, we will receive an update via the support ticket. Hence closing this issue.

IBM-Cloud / terraform-provider-ibm