Bug: terraform fails prematurely when switching from whitelisted_ips to nsg_ids using oci_database_autonomous_database

luckeyca commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version and Provider Version

All

Affected Resource(s)

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. 
# Please remove any sensitive information from configuration files before sharing them. 

module "adb_toronto" {
  source                    = "../../.."
  providers                 = { oci = oci.toronto }
  deploy                    = var.deploy
  compartment_ocid          = var.compartment_ocid
  name                      = format("%s%s", local.db_base_name, "to")
  display_name              = var.toronto_display_name
  db_workload               = var.toronto_db_workload
  adb_source                = var.adb_source
  subnet_id                 = var.toronto_subnet_id
  customer_contacts         = var.customer_contacts
  freeform_tags             = merge(local.base_tags, var.toronto_freeform_tags)
  nsg_ids                   = var.toronto_nsg_ids
  #whitelisted_ips           = var.whitelisted_ips
}

Expected Behavior

When switching from using whitelisted_ips(Secure access from allowed IPs and VCNs only option from console) to using nsg_ids(Private endpoint access only), since the "terraform plan" works WITHOUT any error, terraform apply should work without any issue.

Actual Behavior

even though terraform plan worked, terraform apply failed(see error below) with error "cannot update ACL due to private endpoint". however, from console, I could see the whitelist was already removed. Also re-run of the "terraform apply" immediately was successful.

The error when the first "terraform apply" failed.

│ Error: 400-InvalidParameter, Cannot update the Autonomous Database's access control list (ACL) because a private endpoint is configured for the database. │ Suggestion: Please update the parameter(s) in the Terraform config as per error message Cannot update the Autonomous Database's access control list (ACL) because a private endpoint is configured for the database. │ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/database_autonomous_database │ API Reference: https://docs.oracle.com/iaas/api/#/en/database/20160918/AutonomousDatabase/UpdateAutonomousDatabase │ Request Target: PUT https://database.ca-toronto-1.oraclecloud.com/20160918/autonomousDatabases/ocid1.autonomousdatabase.oc1.ca-toronto-1.an2g6ljr5xeuumqaqlxeaeuch2shzcyone2fdf4enauwkqujr63nlnyunjra │ Provider version: 4.110.0, released on 2023-03-01. This provider is 1 Update(s) behind to current. │ Service: Database Autonomous │ Operation Name: UpdateAutonomousDatabase │ OPC request ID: fdcbc01e2fa8fa59b007a447c055445b/2594BA9673192153E8D05819D01167F9/F07561E26CBFB339DA54D9D1AA925F2D │ │ │ with module.adb_toronto.oci_database_autonomous_database.primary[0], │ on ../../../main.tf line 21, in resource "oci_database_autonomous_database" "primary": │ 21: resource "oci_database_autonomous_database" "primary" { │ ╵ ╷ │ Error: 400-InvalidParameter, Cannot update the Autonomous Database's access control list (ACL) because a private endpoint is configured for the database. │ Suggestion: Please update the parameter(s) in the Terraform config as per error message Cannot update the Autonomous Database's access control list (ACL) because a private endpoint is configured for the database. │ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/database_autonomous_database │ API Reference: https://docs.oracle.com/iaas/api/#/en/database/20160918/AutonomousDatabase/UpdateAutonomousDatabase │ Request Target: PUT https://database.ca-montreal-1.oraclecloud.com/20160918/autonomousDatabases/ocid1.autonomousdatabase.oc1.ca-montreal-1.an4xkljr5xeuumqaeyy234ewtpyy5jd7pa2ho7zklgtghgsevlb55ydoi4za │ Provider version: 4.110.0, released on 2023-03-01. This provider is 1 Update(s) behind to current. │ Service: Database Autonomous │ Operation Name: UpdateAutonomousDatabase │ OPC request ID: 4c15c4053d68fcdc82376aaf76a969a5/DC0FE9588B9BE4D7F7E911A149C4AB3E/10E5942060C8D3DABD3B65A642AA70A9 │ │ │ with module.adb_montreal.oci_database_autonomous_database.primary[0], │ on ../../../main.tf line 21, in resource "oci_database_autonomous_database" "primary": │ 21: resource "oci_database_autonomous_database" "primary" {

Steps to Reproduce

create an adb using oci_database_autonomous_database resource(don't need module) and use whitelisted_ips argument, but do NOT use subnet_id and nsg_ids arguments. This is equivalent of the "Secure access from allowed IPs and VCNs only" option from oci console.
after adb up and running. comment out the whitelisted_Ips argument, add subnet_id to a private subnet along with the nsg_ids associated with the vcn where the private subnet located.
run "terraform plan". This will succeed without issue and shows the correct changes(removing whitelisted_ips/ACL and adding subnet/private endpoint/nsg_ids).
run "terraform apply". It will fail near to the end with error "cannot update ACL due to private endpoint is configured"
check the oci console, you should see actually all the changes were done successfully already.
re-run "terraform apply". it will run successfully this time and only changes are the new terraform private endpoint outputs.

Important Factoids

References

luckeyca commented 1 year ago

The same issue happens when switching from using nsg_ids to whitelisted_ips, but failed much earlier. same steps to reproduce the issue. Judging from the issue, there are two problems here;

terraform plan doesn't have the necessary application(database) level input validations. I have another ticket for this type of the issues as this happens for many arguments with dependency conditions. https://github.com/oracle/terraform-provider-oci/issues/1791
terraform apply using the api, but the api was changing to private endpoint before attempting to update ACL which caused the issue.