[COS] Lifecycle Configuration not fully complete on terraform apply

jor2 commented 2 weeks ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

Affected Resource(s)

ibm_cos_bucket_lifecycle_configuration

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

main.tf

##############################################################################
# Resource Group
##############################################################################

module "resource_group" {
  source  = "terraform-ibm-modules/resource-group/ibm"
  version = "1.1.6"
  # if an existing resource group is not set (null) create a new one using prefix
  resource_group_name          = var.resource_group == null ? "${var.prefix}-resource-group" : null
  existing_resource_group_name = var.resource_group
}

##############################################################################
# Create Cloud Object Storage instance and a bucket
##############################################################################

resource "ibm_resource_instance" "cos" {
  name              = "${var.prefix}-cos"
  resource_group_id = module.resource_group.resource_group_id
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
  tags              = []
}

##############################################################################
# Create Cloud Object Storage bucket using buckets submodule
##############################################################################

resource "ibm_cos_bucket" "cos_bucket" {
  bucket_name           = "${var.prefix}-bucket-module"
  resource_instance_id  = ibm_resource_instance.cos.id
  region_location       = var.region
  endpoint_type         = "public"
  storage_class         = "standard"
  hard_quota            = null
  force_delete          = true
  object_lock           = null
}

resource "ibm_cos_bucket_lifecycle_configuration" "cos_bucket_lifecycle" {
  bucket_crn      = ibm_cos_bucket.cos_bucket.crn
  bucket_location = ibm_cos_bucket.cos_bucket.region_location

  lifecycle_rule {
    expiration {
      days = 365
    }
    filter {
      prefix = ""
    }
    rule_id = "expiry-rule"
    status  = "enable"
  }

  lifecycle_rule {
    transition {
      days = 90
      storage_class = "GLACIER"
    }
    filter {
      prefix = ""
    }
    rule_id = "archive-rule"
    status  = "enable"
  }
}

variables.tf

variable "ibmcloud_api_key" {
  type        = string
  description = "The IBM Cloud API Token"
  sensitive   = true
}

variable "prefix" {
  type        = string
  description = "Prefix name for all related resources"
  default = "test-cos"
}

variable "resource_tags" {
  type        = list(string)
  description = "Optional list of tags to be added to created resources"
  default     = []
}

variable "region" {
  description = "Region where resources will be created"
  type        = string
  default = "us-south"
}

variable "resource_group" {
  type        = string
  description = "An existing resource group name to use for this example, if unset a new resource group will be created"
  default     = null
}

providers.tf

provider "ibm" {
  ibmcloud_api_key = var.ibmcloud_api_key
}

version.tf

terraform {
  required_version = ">= 1.4.0"

  # Ensure that there is always 1 example locked into the lowest provider version of the range defined in the main
  # module's version.tf (this example), and 1 example that will always use the latest provider version (advanced and fscloud examples).
  required_providers {
    ibm = {
      source  = "ibm-cloud/ibm"
      version = "1.70.0"
    }
  }
}

outputs.tf

output "bucket_name" {
  description = "Bucket name"
  value       = "${var.prefix}-bucket-module"
}

output "bucket_crn" {
  description = "Bucket CRN"
  value       = ibm_cos_bucket.cos_bucket.crn
}

output "bucket_id" {
  description = "Bucket id"
  value       = ibm_cos_bucket.cos_bucket.id
}

Debug Output

logs.txt

Panic Output

Planning failed. Terraform encountered an error while generating this plan.
╷
│ Error: [ERROR] Error getting Lifecycle Configuration for the bucket test-cos-bucket-module
│ 
│   with ibm_cos_bucket_lifecycle_configuration.cos_bucket_lifecycle,
│   on main.tf line 42, in resource "ibm_cos_bucket_lifecycle_configuration" "cos_bucket_lifecycle":
│   42: resource "ibm_cos_bucket_lifecycle_configuration" "cos_bucket_lifecycle" {

Expected Behavior

The plan should not have failed due to not being able to fetch the Lifecycle Configuration that was just created.

Actual Behavior

It fails to fetch the lifecyle configuration. If you give it a few more seconds and run the plan it will pass. This suggests the lifecyle policy is not fully complete yet being marked as complete by terraform. Adding a 30second sleep on create fixes this.

Error getting Lifecycle Configuration for the bucket test-cos-bucket-module

Steps to Reproduce

terraform apply
terraform plan ran instantly after the apply completes.

Important Factoids

References

0000

IBM-diksha commented 2 weeks ago

Thank you @jor2 for reaching out. I am looking into this one. And will get back to you on this one.

IBM-diksha commented 3 days ago

@jor2 This is caused due to the lag between the request to create the lifecycle configuration and it getting applied at the backend. We are checking with the backend team on this one and get back to you, mean while a delay between the terraform apply could be used as a work around. Thank you.

jor2 commented 2 days ago

@jor2 This is caused due to the lag between the request to create the lifecycle configuration and it getting applied at the backend. We are checking with the backend team on this one and get back to you, mean while a delay between the terraform apply could be used as a work around. Thank you.

@ocofaigh should we wait for the fix? We can just implement the sleep and then remove once its done instead of having to maintain the pr with future commits

ocofaigh commented 2 days ago

@IBM-diksha what is your plan for fix? If the backend is still applying config, I would expect the fix should be that the ibm_cos_bucket_lifecycle_configuration resource should not be marked as complete by terraform until the config is indeed set on the backend. Perhaps you can update the provider code to do some kind of GET before marking resource as complete?

@jor2 OK I guess we can proceed with the sleep workaround in our PR

IBM-Cloud / terraform-provider-ibm