search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.8k stars 9.15k forks source link

[Bug]: "aws_db_instance", "storage_type" (io1 > gp3) can't be changed if "allocated_storage" is under 400 GB #28589

Open crimean-celica opened 1 year ago

crimean-celica commented 1 year ago

Terraform Core Version

1.0.9

AWS Provider Version

4.48.0

Affected Resource(s)

aws_db_instance

Expected Behavior

"storage_type" can be changed from "io1" to "gp3"

Actual Behavior

"storage_type" can't be changed (io1 > gp3) as far as you have under 400 GB disk

if you do not specify "iops":

Error: updating RDS DB Instance (delme2): operation error RDS: ModifyDBInstance, https response error StatusCode: 400, RequestID: 20ed3bd9-1794-4451-a314-6eff87d6e851, api error InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.

if you do specify "iops":

Error: updating RDS DB Instance (inapp-stage-delme2): operation error RDS: ModifyDBInstance, https response error StatusCode: 400, RequestID: 44cad8d2-dea0-4c77-9979-2314cc8d528a, api error InvalidParameterCombination: You can't specify IOPS or storage throughput for engine postgres and a storage size less than 400.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_db_instance" "delme2" {
  identifier = "delme2"

  engine            = "postgres"
  engine_version    = "13.7"
  instance_class    = "db.t4g.small"
  allocated_storage = 100
  storage_encrypted = true
  apply_immediately = true
  storage_type      = "io1"
  iops              = 1000

  iam_database_authentication_enabled = true

  username = "delme_db_admin"
  password = "P@ssw0rd"
  port     = "5432"

  parameter_group_name = "default.postgres12"

  deletion_protection = false
}

Steps to Reproduce

  1. create a new RDS instance ( storage_type = "io1", allocated_storage = 100, iops = 1000 )
  2. change storage_type to "gp3" with or without specifying "iops" value

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 1 year ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

crimean-celica commented 1 year ago

Technically, it's possible to change io1 > gp2 > gp3 (it works, I tested) but this way will have gp2 (with lower iops) during ~6 hours (storage optimization time).

fatmcgav-depop commented 1 year ago

I was able to reproduce this today when moving a 5TB RDS instance from io1 to gp3.

2023-01-10T10:54:54.771Z [INFO]  provider.terraform-provider-aws_v4.46.0: [INFO] Only settings updating, instance changes will be applied in next maintenance window
2023-01-10T10:54:54.771Z [DEBUG] provider.terraform-provider-aws_v4.46.0: [DEBUG] Waiting for state to become: [success]
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: [DEBUG] [aws-sdk-go-v2] Request
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: POST / HTTP/1.1
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Host: rds.us-east-1.amazonaws.com
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.2.7 (+https://www.terraform.io) terraform-provider-aws/4.46.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.17.2 os/macos lang/go/1.19.3 md/GOOS/darwin md/GOARCH/amd64 api/rds/1.32.0
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Content-Length: 160
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Amz-Sdk-Invocation-Id: 5e2b277b-4365-41c0-8cf9-addcf1e79007
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Amz-Sdk-Request: attempt=1; max=25
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Authorization: AWS4-HMAC-SHA256 Credential=ASIA47JNX4DM3MJWQSEB/20230110/us-east-1/rds/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=**redacted**
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Content-Type: application/x-www-form-urlencoded
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: X-Amz-Date: 20230110T105454Z
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: X-Amz-Security-Token: **redacted**
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Accept-Encoding: gzip
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0:
2023-01-10T10:54:54.772Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Action=ModifyDBInstance&DBInstanceIdentifier=**redacted**&DeletionProtection=true&StorageThroughput=2000&StorageType=gp3&Version=2014-10-31
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: [DEBUG] [aws-sdk-go-v2] Response
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: HTTP/1.1 400 Bad Request
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Connection: close
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Content-Length: 377
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Content-Type: text/xml
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Date: Tue, 10 Jan 2023 10:54:55 GMT
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: Strict-Transport-Security: max-age=31536000
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: X-Amzn-Requestid: 646d7ca3-764f-47bb-be88-9d171bc06714
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: <ErrorResponse xmlns="http://rds.amazonaws.com/doc/2014-10-31/">
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:   <Error>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:     <Type>Sender</Type>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:     <Code>InvalidParameterCombination</Code>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:     <Message>You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.</Message>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:   </Error>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0:   <RequestId>646d7ca3-764f-47bb-be88-9d171bc06714</RequestId>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: </ErrorResponse>
2023-01-10T10:54:55.214Z [DEBUG] provider.terraform-provider-aws_v4.46.0: [DEBUG] [aws-sdk-go-v2] request failed with unretryable error https response error StatusCode: 400, RequestID: 646d7ca3-764f-47bb-be88-9d171bc06714, api error InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.

I think this boils down to a logic flaw here: https://github.com/hashicorp/terraform-provider-aws/blob/1076f598ee88175e7409c5887edcf87e6cbeab20/internal/service/rds/instance.go#L2065-L2072

ipolishchuk commented 1 year ago

I have the same problem with terraform 1.3.6 provider hashicorp/aws v4.49.0

ipolishchuk commented 1 year ago

╷ │ Error: updating RDS DB Instance (XXXXXXXXX): operation error RDS: ModifyDBInstance, https response error StatusCode: 400, RequestID: 4cb24468-753e-4397-8364-6fdd7cad2977, api error InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops. │ │ with module.db_instance-0.module.db_instance.module.db_instance.aws_db_instance.this[0], │ on ../../../../../modules/terraform-aws-rds/5.2.2/modules/db_instance/main.tf line 32, in resource "aws_db_instance" "this": │ 32: resource "aws_db_instance" "this" { │

fatmcgav commented 1 year ago

I've opened #28847 with a change that I think should solve this issue.

Unfortunately I'm unable to run the acceptance tests atm, but will see if I can work out a way to get them run soon...

sindhu-selvaraj-starling commented 1 year ago

This is an issue with AWS CLI command, I am contacting their support now

aws rds modify-db-instance --db-instance-identifier database-1 --storage-type gp3 --apply-immediately                   

An error occurred (InvalidParameterCombination) when calling the ModifyDBInstance operation: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.
sindhu-selvaraj-starling commented 1 year ago

When allocated_storage is specified, it allows to do the change via CLI

aws rds modify-db-instance --db-instance-identifier  database-2 --storage-type gp3 --allocated-storage 100 --apply-immediately