[ICD][Postgresql] Error updating database auto_scaling: Unprocessable Entity

ifs-anthonylecarrer commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

terraform : 1.3 terraform ibmcloud provider : 1.50.0

Affected Resource(s)

ibm_database

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

variable "resource_group" {
  type        = string
  description = "Name of the resource group"
}
variable "lz_resource_prefix" {
  type        = string
  default     = "sbx"
  description = "Prefix of the landing zone managed by Sandbox team"
}
variable "lz_resource_group" {
  type        = string
  default     = "rg-sandbox"
  description = "Resource Group of the landing zone managed by Sandbox team"
}
variable "region" {
  description = "Location where the bucket will be provisioned."
  default     = "eu-fr2"
  type        = string
}
variable "database_plan" {
  type        = string
  description = "Resource instance name for example 'my-pg-database'"
  default = "standard"
}
variable "database_instance_name" {
  type        = string
  description = "Resource instance name for example 'my-pg-database'"
}
variable "database_version" {
  default     = null
  type        = string
  description = "The database version to provision if specified, latest if not specified. Warning."
}
variable "memory_allocation" {
  default     = 1024
  type        = number
  description = "Memory allocation required for database"
}
variable "disk_allocation" {
  default     = 61440
  type        = number
  description = "Disk allocation required for database"
}
variable "cpu_allocation" {
  default     = 0
  type        = number
  description = "CPU allocation required for database"
}
variable "service_endpoints" {
  default     = "private"
  type        = string
  description = "Types of the service endpoints. Possible values are 'public', 'private', 'public-and-private'."
}
variable "allowedlistips" {
  default     = null
  type        = set(map(string))
  description = "Allowed ips to access database"
}
variable "cpu_rate_increase_percent" {
  default     = null
  type        = number
  description = "Auto Scaling CPU Rate: Increase Percent"
}
variable "cpu_rate_limit_count_per_member" {
  default     = null
  type        = number
  description = "Auto Scaling CPU Rate: Limit count per number"
}
variable "cpu_rate_period_seconds" {
  default     = null
  type        = number
  description = "Auto Scaling CPU Rate: Period Seconds"
}
variable "cpu_rate_units" {
  default     = null
  type        = string
  description = "Auto Scaling CPU Rate: Units "
}
variable "disk_capacity_enabled" {
  default     = null
  type        = bool
  description = "Auto Scaling Disk Scalar: Capacity Enabled"
}
variable "disk_free_space_less_than_percent" {
  default     = null
  type        = number
  description = "Auto Scaling Disk Scalar: Capacity Free Space Less Than Percent"
}
variable "disk_io_above_percent" {
  default     = null
  type        = number
  description = "Auto Scaling Disk Scalar: IO Utilization Above Percent"
}
variable "disk_io_enabled" {
  default     = null
  type        = bool
  description = "Auto Scaling Disk Scalar: IO Utilization Enabled"
}
variable "disk_io_over_period" {
  default     = null
  type        = string
  description = "Auto Scaling Disk Scalar: IO Utilization Over Period"
}
variable "disk_rate_increase_percent" {
  default     = null
  type        = number
  description = "Auto Scaling Disk Rate: Increase Percent"
}
variable "disk_rate_limit_mb_per_member" {
  default     = null
  type        = number
  description = "Auto Scaling Disk Rate: Limit mb per member"
}
variable "disk_rate_period_seconds" {
  default     = null
  type        = number
  description = "Auto Scaling Disk Rate: Period Seconds"
}
variable "disk_rate_units" {
  default     = null
  type        = string
  description = "Auto Scaling Disk Rate: Units "
}
variable "memory_io_above_percent" {
  default     = null
  type        = number
  description = "Auto Scaling Memory Scalar: IO Utilization Above Percent"
}
variable "memory_io_enabled" {
  default     = null
  type        = bool
  description = "Auto Scaling Memory Scalar: IO Utilization Enabled"
}
variable "memory_io_over_period" {
  default     = null
  type        = number
  description = "Auto Scaling Memory Scalar: IO Utilization Over Period"
}
variable "memory_rate_increase_percent" {
  default     = null
  type        = number
  description = "Auto Scaling Memory Rate: Increase Percent"
}
variable "memory_rate_limit_mb_per_member" {
  default     = null
  type        = number
  description = "Auto Scaling Memory Rate: Limit mb per member"
}
variable "memory_rate_period_seconds" {
  default     = null
  type        = number
  description = "Auto Scaling Memory Rate: Period Seconds"
}
variable "memory_rate_units" {
  default     = null
  type        = string
  description = "Auto Scaling Memory Rate: Units "
}
variable "tags" {
  default     = null
  type        = set(string)
  description = "Tags for the database"
}

resource "ibm_database" "database_postgresql" {
  resource_group_id                    = data.ibm_resource_group.resource_group.id
  name                                 = var.database_instance_name
  service                              = "databases-for-postgresql"
  plan                                 = var.database_plan
  location                             = var.region
  version                              = (var.database_version != null ? var.database_version : null)
  group {
    group_id = "member"
    memory { 
      allocation_mb = var.memory_allocation
    }
    disk { 
      allocation_mb = var.disk_allocation 
    }
    cpu {
      allocation_count = var.cpu_allocation
    }
  }
  service_endpoints                    = "private"
  key_protect_instance                 = ibm_resource_instance.kms_instance.guid
  key_protect_key                      = data.ibm_kms_keys.encryption_data.keys.0.crn
  backup_encryption_key_crn            = data.ibm_kms_keys.backup_encryption_data.keys.0.crn
  tags                                 = (var.tags != null ? var.tags : [])
  dynamic "whitelist" {
    for_each = var.allowedlistips != null ? var.allowedlistips : []
    content {
      address     = format("%s/32", whitelist.value)
      description = whitelist.value
    }
  }
  auto_scaling {
    cpu {
      rate_increase_percent       = (var.cpu_rate_increase_percent != null ? var.cpu_rate_increase_percent : null)
      rate_limit_count_per_member = (var.cpu_rate_limit_count_per_member != null ? var.cpu_rate_limit_count_per_member : null)
      rate_period_seconds         = (var.cpu_rate_period_seconds != null ? var.cpu_rate_period_seconds : null)
      rate_units                  = (var.cpu_rate_units != null ? var.cpu_rate_units : null)
    }
    disk {
      capacity_enabled             = (var.disk_capacity_enabled != null ? var.disk_capacity_enabled : null)
      free_space_less_than_percent = (var.disk_free_space_less_than_percent != null ? var.disk_free_space_less_than_percent : null)
      io_above_percent             = (var.disk_io_above_percent != null ? var.disk_io_above_percent : null)
      io_enabled                   = (var.disk_io_enabled != null ? var.disk_io_enabled : null)
      io_over_period               = (var.disk_io_over_period != null ? var.disk_io_over_period : null)
      rate_increase_percent        = (var.disk_rate_increase_percent != null ? var.disk_rate_increase_percent : null)
      rate_limit_mb_per_member     = (var.disk_rate_limit_mb_per_member != null ? var.disk_rate_limit_mb_per_member : null)
      rate_period_seconds          = (var.disk_rate_period_seconds != null ? var.disk_rate_period_seconds : null)
      rate_units                   = (var.disk_rate_units != null ? var.disk_rate_units : null)
    }
    memory {
      io_above_percent         = (var.memory_io_above_percent != null ? var.memory_io_above_percent : null)
      io_enabled               = (var.memory_io_enabled != null ? var.memory_io_enabled : null)
      io_over_period           = (var.memory_io_over_period != null ? var.memory_io_over_period : null)
      rate_increase_percent    = (var.memory_rate_increase_percent != null ? var.memory_rate_increase_percent : null)
      rate_limit_mb_per_member = (var.memory_rate_limit_mb_per_member != null ? var.memory_rate_limit_mb_per_member : null)
      rate_period_seconds      = (var.memory_rate_period_seconds != null ? var.memory_rate_period_seconds : null)
      rate_units               = (var.memory_rate_units != null ? var.memory_rate_units : null)
    }
  }
}

Debug Output

2023/02/07 19:17:03 Terraform apply | ibm_database.database_postgresql: Still creating... [6m50s elapsed]
 2023/02/07 19:17:13 Terraform apply | ibm_database.database_postgresql: Still creating... [7m0s elapsed]
 2023/02/07 19:17:21 Terraform apply | 
 2023/02/07 19:17:21 Terraform apply | Error: [ERROR] Error updating database auto_scaling: Unprocessable Entity
 2023/02/07 19:17:21 Terraform apply | 
 2023/02/07 19:17:21 Terraform apply |   with ibm_database.database_postgresql,
 2023/02/07 19:17:21 Terraform apply |   on main.tf line 59, in resource "ibm_database" "database_postgresql":
 2023/02/07 19:17:21 Terraform apply |   59: resource "ibm_database" "database_postgresql" {
 2023/02/07 19:17:21 Terraform apply | 
 2023/02/07 19:17:21 [1m[31mTerraform APPLY error: Terraform APPLY errorexit status 1[39m[0m

Panic Output

Expected Behavior

NO ERROR

Actual Behavior

Deploy failed

Steps to Reproduce

terraform apply -var-file validation-values.json

{
    "region": "eu-fr2",
    "resource_group": "rg-dev",
    "database_instance_name": "simple-test",
    "tags": ["test"]
}

Important Factoids

References

0000

omaraibrahim commented 1 year ago

@ifs-anthonylecarrer Thank you for reaching out! Could you please set the logs to debug mode? You can do so as follows in the Terraform Provider, export TF_LOG=DEBUG

After doing so, please run your code again. You should hopefully see all the API calls being triggered in the logs. Then send them and we'll be able to better assist you :D

Thank you and I apologize for the inconvenience.

ifs-anthonylecarrer commented 1 year ago

The provider 1.49 is working with the same code. Are u asking me to do your job and to test your provider ?

omaraibrahim commented 1 year ago

@ifs-anthonylecarrer I'm asking you to provide the logs to help me support you.

alexhemard commented 1 year ago

@ifs-anthonylecarrer auto_scaling was refactored to use a different client library in the last release so this does appear to be a change in behavior. The template provided is setting auto_scaling blocks with null values which now in v1.50.0 results in an API call that returns an error. Here is the autoscaling API call/response from the template you provided.

PATCH https://api.us-south.databases.cloud.ibm.com/v5/ibm/deployments/{crn}/groups/member/autoscaling

{
  "autoscaling": {
    "disk": {
      "scalers": {
        "capacity": {
          "enabled": false,
          "free_space_less_than_percent": 0
        },
        "io_utilization": {
          "enabled": false,
          "over_period": "",
          "above_percent": 0
        }
      },
      "rate": {
        "increase_percent": 0,
        "period_seconds": 0,
        "limit_mb_per_member": 0,
        "units": ""
      }
    },
    "memory": {
      "scalers": {
        "io_utilization": {
          "enabled": false,
          "over_period": "",
          "above_percent": 0
        }
      },
      "rate": {
        "increase_percent": 0,
        "period_seconds": 0,
        "limit_mb_per_member": 0,
        "units": ""
      }
    }
  }
}

Response

HTTP/2.0 422 Unprocessable Entity

{
  "errors": {
    "autoscaling.rate.period_seconds": [
      "must be greater than or equal to 60"
    ],
    "disk.scalers.io_utilization.over_period": [
      "must be one of [\"5m\", \"15m\", \"30m\", \"60m\"]" a user error
    ]
  }
}

I would suggest as an interim fix to modify the template to not set auto_scaling blocks with null / empty values as this has undocumented behavior while we investigate. Also, the cpu auto_scaling block is unused and can be removed as there is no auto scaling for CPU resources for IBM Cloud Databases.

shemau commented 1 year ago

There are additional issues with the cpu block in IBM Cloud provider 1.51 and 1.53.

When setting the auto_scaling cpu rate_limit_count_per_member to 20 the first apply works, but fails to set the value. This is clearly a bug.

Folloing this a second apply (with no changes) results in:

Terraform will perform the following actions:

  # module.postgresql_db.ibm_database.postgresql_db will be updated in-place
  ~ resource "ibm_database" "postgresql_db" {
        id                           = "crn:v1:bluemix:public:databases-for-postgresql:us-south:a/abac0df06b644a9cabc6e44f55b3880e:454508e1-4d6a-45e3-aac1-98c08353ab6f::"
        name                         = "steve-n12-postgres"
        tags                         = []
        # (26 unchanged attributes hidden)

      ~ auto_scaling {
          ~ cpu {
              ~ rate_limit_count_per_member = 30 -> 20
                # (3 unchanged attributes hidden)
            }

            # (2 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.postgresql_db.ibm_database.postgresql_db: Modifying... [id=crn:v1:bluemix:public:databases-for-postgresql:us-south:a/abac0df06b644a9cabc6e44f55b3880e:454508e1-4d6a-45e3-aac1-98c08353ab6f::]
╷
│ Error: [ERROR] Error updating database memory auto_scaling group: Unprocessable Entity
│   
│   with module.postgresql_db.ibm_database.postgresql_db,
│   on ../../main.tf line 42, in resource "ibm_database" "postgresql_db":
│   42: resource "ibm_database" "postgresql_db" {
│ 
╵

which is another bug, since it was processable the first time and is suddenly no longer processable.

Should the documentation for the cpu block be removed from the terraform ibm_database resource if it is not used and causes issues if is coded? Additionally the documentation is missing an entry for auto_scaling disk io_over_period which appears to be a legitimate value with a counterpart in the UI (Resource, Autoscaling, disk average IO utilization above, for more than box).

ocofaigh commented 1 year ago

Just to add to @shemau's comment above, when you try and actually set the rate_limit_count_per_member to 30 in the code, the error you get is:

│ Error: [ERROR] Error updating database cpu auto_scaling group: Request failed with status code: 422, ServerErrorResponse: {"errors":{"cpu.rate.limit_count_per_member":["cannot autoscale higher than regular scaling limit"]}}
│ 
│   with module.postgresql_db.ibm_database.postgresql_db,
│   on ../../main.tf line 42, in resource "ibm_database" "postgresql_db":
│   42: resource "ibm_database" "postgresql_db" {
│ 
╵

If CPU auto scaling is not supported by ICD, then the documentation needs to be updated because clearly if you try to use it, things break. cc @alexhemard I would also expect the provider to do some validation here before it attempts to use these values during provision time, and hence fails the terraform apply. Should be caught at plan time

alexhemard commented 1 year ago

@ocofaigh Correct, there is no cpuautoscaling. Our docs do not explicity state it, but confusingly it appears in our API. We are looking at clarifying/removing the cpu block.

Additionally the documentation is missing an entry for auto_scaling disk io_over_period which appears to be a legitimate value with a counterpart in the UI

@shemau Thanks for catching this! I'll try and get this fix prioritized

IBM-Cloud / terraform-provider-ibm