search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.83k stars 9.17k forks source link

Terraform not able to set IOPS value while modifying RDS red replica if we are enalinbling max_storage for storage_type "io1"( PostgreSQL ) #11454

Open ghost opened 4 years ago

ghost commented 4 years ago

This issue was originally opened by @sravankurra as hashicorp/terraform#23764. It was migrated here as a result of the provider split. The original body of the issue is below.

Terraform Version

Terraform v0.12.1

Terraform Configuration Files

resource "aws_db_instance" "master" {
  provider                                 = "aws.master_region"
  apply_immediately                 = "true"
  allocated_storage                   = "${var.allocated_storage}"
  max_allocated_storage           = "${var.max_allocated_storage}"
  auto_minor_version_upgrade   = "false"
  engine                                      = "${var.engine}"
  engine_version                          = "${var.engine_version_master}"
  instance_class                           = "${var.instance_class}"
  identifier                                      = "${var.instance_identifier}-${var.cluster_version}"
  name                                           = "${var.name}"
  port                                              = "5432"
  username                                     = "${data.vault_generic_secret.rundeck_auth.data["admin_username"]}"
  password                                       = "${data.vault_generic_secret.rundeck_auth.data["admin_password"]}"
  multi_az                                           = "true"
  enabled_cloudwatch_logs_exports = ["postgresql", "upgrade"]
  backup_window                               = "06:00-09:00"
  iops                                                  = "${var.iops}"
}

resource "aws_db_instance" "master_region_replicas" {
  provider                        = "aws.master_region"
  count                           = "${var.local_replica_count}"
  apply_immediately               = "true"
  allocated_storage               = "${var.allocated_storage}"
  max_allocated_storage           = "${var.max_allocated_storage}"
  auto_minor_version_upgrade      = "false"
  engine                          = "${var.engine}"
  engine_version                  = "${var.engine_version_local_replicas}"
  instance_class                  = "${var.instance_class}"
  identifier                      = "${var.instance_identifier}-replica-${count.index + 1}"
  name                            = "${var.name}"
  port                            = "5432"
  parameter_group_name            = "${aws_db_instance.master.parameter_group_name}"
  iops                            = "${var.iops}"
  replicate_source_db = "${aws_db_instance.master.id}"
}
resource "aws_db_instance" "cross_region_replicas" {
  provider                        = "aws.cross_region"
  count                           = "${var.cross_replica_count}"
  apply_immediately               = "true"
  allocated_storage               = "${var.allocated_storage}"
  max_allocated_storage           = "${var.max_allocated_storage}"
  auto_minor_version_upgrade      = "false"
storage_type        = "${var.storage_type}"
  replicate_source_db = "${aws_db_instance.master.arn}"
 iops                            = "${var.iops}"
}

Debug Output

Form my debug out put log I observed terraform plugin action not able to pass Iops value. that causing error in terrafor out put and not able to enable auto scaling in RDS rerplicas in AWS

2019-12-27T17:34:02.505-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:02 [DEBUG] DB Instance status for instance testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1: modifying
2019-12-27T17:34:02.505-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:02 [TRACE] Waiting 10s before next try
2019/12/27 17:34:05 [TRACE] dag/walk: vertex "module.postgresql.provider.aws.cross_region (close)" is waiting for "module.postgresql.aws_db_instance.cross_region_replicas[0]"
2019/12/27 17:34:05 [TRACE] dag/walk: vertex "module.postgresql.output.rds_cross_repls_id" is waiting for "module.postgresql.aws_db_instance.cross_region_replicas[0]"
2019/12/27 17:34:05 [TRACE] dag/walk: vertex "module.postgresql.output.rds_cross_repls_address" is waiting for "module.postgresql.aws_db_instance.cross_region_replicas[0]"
2019/12/27 17:34:07 [TRACE] dag/walk: vertex "meta.count-boundary (EachMode fixup)" is waiting for "module.postgresql.output.rds_cross_repls_id"
2019/12/27 17:34:07 [TRACE] dag/walk: vertex "root" is waiting for "module.postgresql.provider.aws.cross_region (close)"
2019/12/27 17:34:10 [TRACE] dag/walk: vertex "module.postgresql.provider.aws.cross_region (close)" is waiting for "module.postgresql.aws_db_instance.cross_region_replicas[0]"
2019/12/27 17:34:10 [TRACE] dag/walk: vertex "module.postgresql.output.rds_cross_repls_id" is waiting for "module.postgresql.aws_db_instance.cross_region_replicas[0]"
2019/12/27 17:34:10 [TRACE] dag/walk: vertex "module.postgresql.output.rds_cross_repls_address" is waiting for "module.postgresql.aws_db_instance.cross_region_replicas[0]"
2019/12/27 17:34:12 [TRACE] dag/walk: vertex "root" is waiting for "module.postgresql.provider.aws.cross_region (close)"
2019/12/27 17:34:12 [TRACE] dag/walk: vertex "meta.count-boundary (EachMode fixup)" is waiting for "module.postgresql.output.rds_cross_repls_id"
2019-12-27T17:34:12.506-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:12 [DEBUG] DB Instance describe configuration: {
2019-12-27T17:34:12.506-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   DBInstanceIdentifier: "testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1"
2019-12-27T17:34:12.506-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: }
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:12 [DEBUG] [aws-sdk-go] DEBUG: Request rds/DescribeDBInstances Details:
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: POST / HTTP/1.1
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Host: rds.us-west-2.amazonaws.com
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: User-Agent: aws-sdk-go/1.26.5 (go1.13.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.1 (+https://www.terraform.io)
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Content-Length: 129
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Authorization: xxxxxx Credential=xxxxxxxxx
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: X-Amz-Date: 20191227T223412Z
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: X-Amz-Security-Token: xxxxxxxxxxxx
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Accept-Encoding: gzip
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Action=DescribeDBInstances&DBInstanceIdentifier=testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1&Version=2014-10-31
2019-12-27T17:34:12.507-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: -----------------------------------------------------
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:13 [DEBUG] [aws-sdk-go] DEBUG: Response rds/DescribeDBInstances Details:
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: ---[ RESPONSE ]--------------------------------------
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: HTTP/1.1 200 OK
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Connection: close
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Transfer-Encoding: chunked
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Content-Type: text/xml
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Date: Fri, 27 Dec 2019 22:34:12 GMT
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Vary: accept-encoding
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: X-Amzn-Requestid: xxxxxxxxxxxx
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: -----------------------------------------------------
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:13 [DEBUG] [aws-sdk-go] <DescribeDBInstancesResponse xmlns="http://rds.amazonaws.com/doc/2014-10-31/">
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   <DescribeDBInstancesResult>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:     <DBInstances>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:       <DBInstance>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <AllocatedStorage>100</AllocatedStorage>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <EnabledCloudwatchLogsExports>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <member>postgresql</member>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <member>upgrade</member>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </EnabledCloudwatchLogsExports>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <ReadReplicaSourceDBInstanceIdentifier>arn:aws:rds:us-east-1:xxxxxx:db:testing-pgsql-terraform-auto-storage-scaling-v1</ReadReplicaSourceDBInstanceIdentifier>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <AssociatedRoles/>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBParameterGroups>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <DBParameterGroup>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <DBParameterGroupName>default.postgres10</DBParameterGroupName>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <ParameterApplyStatus>in-sync</ParameterApplyStatus>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           </DBParameterGroup>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </DBParameterGroups>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <AvailabilityZone>us-west-2a</AvailabilityZone>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBSecurityGroups/>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <Iops>1000</Iops>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <StatusInfos>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <DBInstanceStatusInfo>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Normal>true</Normal>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <StatusType>read replication</StatusType>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Status>replicating</Status>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           </DBInstanceStatusInfo>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </StatusInfos>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <EngineVersion>10.9</EngineVersion>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <MasterUsername>rds_test_admin</MasterUsername>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <InstanceCreateTime>2019-12-27T22:31:44.161Z</InstanceCreateTime>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBInstanceClass>db.m4.xlarge</DBInstanceClass>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <HttpEndpointEnabled>false</HttpEndpointEnabled>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <ReadReplicaDBInstanceIdentifiers/>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <MonitoringInterval>0</MonitoringInterval>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBInstanceStatus>available</DBInstanceStatus>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <BackupRetentionPeriod>0</BackupRetentionPeriod>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <KmsKeyId>arn:aws:kms:us-west-2:xxxxxxx:key/xxxxxxxxx</KmsKeyId>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <OptionGroupMemberships>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <OptionGroupMembership>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <OptionGroupName>default:postgres-10</OptionGroupName>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Status>in-sync</Status>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           </OptionGroupMembership>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </OptionGroupMemberships>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <CACertificateIdentifier>xxxxx</CACertificateIdentifier>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DbInstancePort>0</DbInstancePort>
2019-12-27T17:34:13.221-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DbiResourceId>db-xxxxxxxx</DbiResourceId>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <PreferredBackupWindow>06:00-09:00</PreferredBackupWindow>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DeletionProtection>false</DeletionProtection>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBInstanceArn>arn:aws:rds:us-west-2:xxxxxx:db:testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1</DBInstanceArn>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBInstanceIdentifier>testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1</DBInstanceIdentifier>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <Endpoint>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <HostedZoneId>xxxxxxx</HostedZoneId>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <Address>testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1.cmfehoos8pzo.us-west-2.rds.amazonaws.com</Address>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <Port>5432</Port>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </Endpoint>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <Engine>postgres</Engine>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <PubliclyAccessible>false</PubliclyAccessible>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <IAMDatabaseAuthenticationEnabled>false</IAMDatabaseAuthenticationEnabled>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <PerformanceInsightsEnabled>false</PerformanceInsightsEnabled>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBName>rds_test</DBName>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <MultiAZ>false</MultiAZ>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DomainMemberships/>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <StorageEncrypted>true</StorageEncrypted>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <DBSubnetGroup>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <VpcId>vpc-xxxxxx</VpcId>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <Subnets>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Subnet>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetIdentifier>subnet-xxxxxx</SubnetIdentifier>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetStatus>Active</SubnetStatus>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetOutpost/>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetAvailabilityZone>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:                 <Name>us-west-2a</Name>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               </SubnetAvailabilityZone>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             </Subnet>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Subnet>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetIdentifier>subnet-xxxxx</SubnetIdentifier>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetStatus>Active</SubnetStatus>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetOutpost/>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetAvailabilityZone>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:                 <Name>us-west-2b</Name>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               </SubnetAvailabilityZone>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             </Subnet>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Subnet>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetIdentifier>subnet-xxxxx</SubnetIdentifier>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetStatus>Active</SubnetStatus>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetOutpost/>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               <SubnetAvailabilityZone>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:                 <Name>us-west-2c</Name>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:               </SubnetAvailabilityZone>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             </Subnet>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           </Subnets>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <SubnetGroupStatus>Complete</SubnetGroupStatus>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <DBSubnetGroupDescription>Common Subnet group for Non-prod RDS in West</DBSubnetGroupDescription>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <DBSubnetGroupName>dre-non-prod-west</DBSubnetGroupName>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </DBSubnetGroup>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <VpcSecurityGroups>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           <VpcSecurityGroupMembership>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <VpcSecurityGroupId>sg-xxxxxx</VpcSecurityGroupId>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:             <Status>active</Status>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:           </VpcSecurityGroupMembership>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         </VpcSecurityGroups>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <LicenseModel>postgresql-license</LicenseModel>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <PendingModifiedValues/>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <PreferredMaintenanceWindow>tue:15:00-tue:16:00</PreferredMaintenanceWindow>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <StorageType>io1</StorageType>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <AutoMinorVersionUpgrade>false</AutoMinorVersionUpgrade>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:         <CopyTagsToSnapshot>false</CopyTagsToSnapshot>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:       </DBInstance>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:     </DBInstances>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   </DescribeDBInstancesResult>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   <ResponseMetadata>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:     <RequestId>xxxxxxxxxx</RequestId>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   </ResponseMetadata>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: </DescribeDBInstancesResponse>
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:13 [DEBUG] DB Instance status for instance testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1: available
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:13 [INFO] DB Instance (testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1) configuration requires ModifyDBInstance: {
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   AllocatedStorage: 100,
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   ApplyImmediately: true,
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   DBInstanceIdentifier: "testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1",
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   DBParameterGroupName: "pg-sravantest-west-rds-test-v1",
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   MaxAllocatedStorage: 200,
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   PreferredMaintenanceWindow: "wed:14:00-wed:15:00",
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   VpcSecurityGroupIds: ["sg-xxxxxxx"]
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: }
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2019/12/27 17:34:13 [DEBUG] [aws-sdk-go] DEBUG: Request rds/ModifyDBInstance Details:
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: POST / HTTP/1.1
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Host: rds.us-west-2.amazonaws.com
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: User-Agent: aws-sdk-go/1.26.5 (go1.13.4; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.1 (+https://www.terraform.io)
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Content-Length: 362
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Authorization: xxxxxxxx Credential=xxxxxxxxxxxxxxxxxxxxxx
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: X-Amz-Date: 20191227T223413Z
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: X-Amz-Security-Token: xxxxxxxxxxx
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Accept-Encoding: gzip
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:
2019-12-27T17:34:13.222-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: Action=ModifyDBInstance&AllocatedStorage=100&ApplyImmediately=true&DBInstanceIdentifier=testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1&DBParameterGroupName=pg-sravantest-west-rds-test-v1&MaxAllocatedStorage=200&PreferredMaintenanceWindow=wed%3A14%3A00-wed%3A15%3A00&Version=2014-10-31&VpcSecurityGroupIds.VpcSecurityGroupId.1=sg-xxxxxx

-->

Crash Output

Error: error modifying DB Instance (testing-pgsql-terraform-auto-storage-scaling-v1-east-replica-1): InvalidParameterCombination: Both storage size and iops must be specified when modifying storage size or iops on an DB Instance that has iops
    status code: 400, request id: xxxxxxxx

  xxxxxx/postgresql.tf line 162, in resource "aws_db_instance" "master_region_replicas":
 162: resource "aws_db_instance" "master_region_replicas" {

Error: error modifying DB Instance (testing-pgsql-terraform-auto-storage-scaling-v1-west-replica-1): InvalidParameterCombination: Both storage size and iops must be specified when modifying storage size or iops on an DB Instance that has iops
    status code: 400, request id: xxxxxx.

Expected Behavior

We should able to create auto scaling in storage for replicas as well, even in log describe instances is showing IOPS value but terraform in not picking IOPS vaulue during modification

Actual Behavior

Steps to Reproduce

Please list the full steps required to reproduce the issue, for example:

  1. terraform init
  2. terraform apply

Additional Context

Not sure why the modify action command is not picking IOPS vaule, is that related providers or logic bug?

References

sravankurra commented 4 years ago

Seems like it is issue with terraform providers if i was disabling auto scaling by passing value as "0" for "io1" I am seeing same terrform error as above ^^ mentioned

sravankurra commented 4 years ago

Test with only giving IOPS variable in my terraform code and removing max_allocated_storage to check this error is only a combination of max_storage and iops, looks like terraform -provider-aws had a issue with passing iops value in modify statement check modify log run by terraform it's not including iops value 

2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: 2020/01/05 14:48:30 [INFO] DB Instance (testing-pgsql-terraform-io1-v1-east-replica-1) configuration requires ModifyDBInstance: {
2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   AllocatedStorage: 100,
2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   ApplyImmediately: true,
2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   DBInstanceIdentifier: "testing-pgsql-terraform-io1-v1-east-replica-1",
2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   DBParameterGroupName: "pg-nonprod-east-rds-test-v1",
2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4:   PreferredMaintenanceWindow: "tue:14:00-tue:15:00"
2020-01-05T14:48:30.700-0500 [DEBUG] plugin.terraform-provider-aws_v2.43.0_x4: }
iarlyy commented 4 years ago

I am not using max_storage, however i've got similar behavior on my env.

My scenario is:

1 - Provisioned a fresh pgsql replica with gp2 storage type 2 - Change storage type from gp2 to io1 + set iops to xxx

Result: nothing happens, read replica is still using gp2

qburst-praven commented 4 years ago

I ran into the same issue today. We specifically set max_allocated_storage to 0 to disable. I tried removing the same too, still same. We use AWS - v2.33 ;

AmundsenJunior commented 4 years ago

I believe I have run into the same issue. I am not able to create an RDS read replica via Terraform and the RDS module. I am adding to this issue here, as my error logs point to the AWS provider's construction of the POST call's parameters in the subsequent ModifyDBInstance action that follows the CreateDBInstance action. Both storage and iops are specified in my inputs to the resource, but only AllocatedStorage is included in the parameters:

2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/27 13:33:27 [DEBUG] DB Instance status for instance rds-replica: available
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/27 13:33:27 [INFO] DB Instance (rds-replica) configuration requires ModifyDBInstance: {
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   AllocatedStorage: 100,
20-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   ApplyImmediately: true,
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   CACertificateIdentifier: "rds-ca-2019",
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   DBInstanceIdentifier: "rds-replica",
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   DBParameterGroupName: "rds-replica-20200723152822693400000004",
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   PreferredBackupWindow: "03:00-06:00",
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   PreferredMaintenanceWindow: "sat:06:00-sat:06:30"
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: }
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/27 13:33:27 [DEBUG] [aws-sdk-go] DEBUG: Request rds/ModifyDBInstance Details:
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: POST / HTTP/1.1
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Host: rds.us-west-2.amazonaws.com
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: User-Agent: aws-sdk-go/1.32.12 (go1.13.7; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.25 (+https://www.terraform.io)
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Length: 327
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Authorization: AWS4-HMAC-SHA256 Credential=key/20200727/us-west-2/rds/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=signature
Type: application/x-www-form-urlencoded; charset=utf-8
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Date: 20200727T173327Z
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Accept-Encoding: gzip
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Action=ModifyDBInstance&AllocatedStorage=100&ApplyImmediately=true&CACertificateIdentifier=rds-ca-2019&DBInstanceIdentifier=rds-replica&DBParameterGroupName=rds-replica-20200723152822693400000004&PreferredBackupWindow=03%3A00-06%3A00&PreferredMaintenanceWindow=sat%3A06%3A00-sat%3A06%3A30&Version=2014-10-31
2020-07-27T13:33:27.065-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: -----------------------------------------------------
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/27 13:33:27 [DEBUG] [aws-sdk-go] DEBUG: Response rds/ModifyDBInstance Details:
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ---[ RESPONSE ]--------------------------------------
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: HTTP/1.1 400 Bad Request
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Connection: close
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Length: 377
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Type: text/xml
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Date: Mon, 27 Jul 2020 17:33:27 GMT
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amzn-Requestid: request-id
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: -----------------------------------------------------
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/27 13:33:27 [DEBUG] [aws-sdk-go] <ErrorResponse xmlns="http://rds.amazonaws.com/doc/2014-10-31/">
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   <Error>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     <Type>Sender</Type>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     <Code>InvalidParameterCombination</Code>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     <Message>You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.</Message>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   </Error>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   <RequestId>request-id</RequestId>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: </ErrorResponse>
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/27 13:33:27 [DEBUG] [aws-sdk-go] DEBUG: Validate Response rds/ModifyDBInstance failed, attempt 0/25, error InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.
2020-07-27T13:33:27.604-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:  status code: 400, request id: request-id
2020/07/27 13:33:27 [DEBUG] module.db.module.db_instance.aws_db_instance.this[0]: apply errored, but we're indicating that via the Error pointer rather than returning it: error modifying DB Instance (analyzer-prod-replica): InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.
    status code: 400, request id: c346463d-0c15-41fb-ab08-3975fe5c11eb
2020/07/27 13:33:27 [ERROR] module.db.module.db_instance: eval: *terraform.EvalApplyPost, err: error modifying DB Instance (analyzer-prod-replica): InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.
    status code: 400, request id: c346463d-0c15-41fb-ab08-3975fe5c11eb
2020/07/27 13:33:27 [ERROR] module.db.module.db_instance: eval: *terraform.EvalSequence, err: error modifying DB Instance (analyzer-prod-replica): InvalidParameterCombination: You must specify both the storage size and iops when modifying the storage size or iops on a DB instance that has iops.
    status code: 400, request id: request-id

Please let me know if this should instead be filed separately with the Terraform RDS module. I found, though, that the module just uses these provider calls, so this seemed most appropriate. I could not, however, find confirmation that the AWS RDS API requires both storage size and iops to always be included in the ModifyDBInstance API call.

justinretzolk commented 2 years ago

Hey y'all 👋 Thank you for taking the time to file this issue, and for the discussion around it! Given that there's been a number of AWS provider releases since the last update, can anyone confirm whether you're still experiencing this behavior?

chrisminton commented 1 year ago

I have received this error with provider version 4.57.1, with a snapshot_identifier, setting gp3 and iops, and using max_storage_allocation = 0

chrisminton commented 1 year ago

In my case, not setting gp3 fixed the issue (I was creating from a snapshot with gp3 already set). It seems unfortunate that explicitly setting this parameter when it won't change the storage type will trigger this issue.