[Bug]: RDS Aurora storage options

[FransUrbo]

Terraform Core Version

v1.3.3

AWS Provider Version

v4.35.0, v4.36.0

Affected Resource(s)

• aws_db_instance

Expected Behavior

Not setting storage option should create a RDS Aurora.

Actual Behavior

TF fails with an error, saying that storage_type is required. AWS documentation say it's not. Same with allocated_storage and iops.

Relevant Error/Panic Output Snippet

Error: provider.aws: aws_db_instance: init: "allocated_storage": required field is not set

Terraform Configuration Files

variable "engine" {
    description                         = "RDS engine type"
    type                                = string
    default                             = "aurora-postgresql"
}

variable "engine_version" {
    description                         = "RDS engine version"
    type                                = string
    default                             = "14.4"
}

resource "aws_db_instance" "main" {
    engine                              = var.engine
    engine_version                      = var.engine_version
   [...]
    # Storage options are not applicable to Aurora.
    storage_type                        = substr(var.engine, 0, 6) != "aurora" ? "io1" : null
    allocated_storage                   = substr(var.engine, 0, 6) != "aurora" ? 300   : null
    iops                                = substr(var.engine, 0, 6) != "aurora" ? 1000  : null

    storage_encrypted                   = substr(var.engine, 0, 6) != "aurora" ? true  : null
    kms_key_id                          = substr(var.engine, 0, 6) != "aurora" ? aws_kms_key.rds.arn : null
}

Steps to Reproduce

Create a aws_db_instance resource without the three storage options.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbinstance.html

Would you like to implement a fix?

No response

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[FransUrbo]

I went back all the way to v2.30.0 (26 Sep 2019) in '10' minor version steps and TF still say that allocated_storage is a required value..

DIdn't bother to go back any further, that's three years of versions! I can't imagine that no-one haven't been able to create a RDS/Aurora* in all that time without complaining, so "something else" must be going on here..

Still think it's wrong that TF say that it's a required value, when it isn't, but how do you create an RDS/Aurora* with TF then?!

[FransUrbo]

Related tickets:

https://github.com/hashicorp/terraform-provider-aws/issues/26092

[ameedDevOps]

I also faced this issue at my staging environment for 8 hours and recreated DB cluster and it's instance result clears all Data form Database and suddenly it reverted and start providing Error: creating RDS Cluster (cluster-name): InvalidParameterCombination: StorageType isn't supported for DB engine aurora-postgresql.

The questions is:

1. Why it is taking automatically some deprecated parameter and
2. If it is taking then why not default value "io1" so that TF will not recreate at least data could be safe.

[jitendra-panchal]

I have also faced this issue on March 28, 2023, where Terraform changed the storage_type variable value to Aurora and recreated the RDS Cluster. In my case, the entire database data is wiped out. This issue should be resolved on a priority basis as it leads to downtime.

[AdamTylerLynch]

Please open a support ticket with AWS support regarding this issue. AWS Support is tracking customer impact.

[taylorsilva]

I don't think this is a bug. I noticed you're using aws_db_instance and I think the answer to this is to use aws_rds_cluster and aws_rds_cluster_instance.

I was able to use Terraform to deploy an RDS cluster using those resource types.

Also, when manually creating an Aurora db I noticed that there aren't really any storage options. You can choose between standard and I/O optimized: I don't know how you pick which one you want with Terraform though.

That said, the docs are very confusing and it's not clear for the aws_rds_cluster resource when the storage values can be set: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster

[bazzuka]

When restoring aurora rds postgres cluster from a snapshot, the storage type is ommited and aurora-iopt1 has no effect.

      + snapshot_identifier          = "aurora-postgres-xxx-prod-2024-08-19-02-14"
      + storage_encrypted            = true
      + storage_type                 = "aurora-iopt1"

The cluster was created with a standard storage type instead. We tested with latest version of AWS provider and got the same results. Can you please look into this?