Upgrade to terraform provider v5.46.0

[mbbush]

Description of your changes

Upgrade to the latest terraform provider version. There have been several complications, discussed below.

I have:

• [x] Read and followed Crossplane's [contribution process].
• [x] Run make reviewable to ensure this PR is ready for review.

Complications

This is integrating several separate pieces, which conflict with each other.

• [ ] AWSClient.Session is now private
  • Since https://github.com/crossplane-contrib/provider-upjet-aws/pull/1241 we're invoking the .Session property on conns.AWSClient
  • In https://github.com/hashicorp/terraform-provider-aws/commit/b87c156fa3ac1199937815266d6f839b509572f3 the terraform AWS provider made the .Session property private (which isn't a breaking change to their public API since it's in internal, but we use a forked version of the TF provider that exposes some internal objects to make our no-fork architecture possible).
  • Thanks to a suggestion from @mergenci I added a getter in my fork of the terraform provider. This appears to be working but I would like help verifying that
  • [ ] metrics are still working properly for both go sdk v1 and v2
• [ ] EKS ClusterAuth
  • In https://github.com/crossplane-contrib/provider-upjet-aws/issues/1248 we observed a regression in the cluster auth generated for an EKS cluster when upgrading to aws go sdk v2 version 1.24.1. We resolved it by pinning the aws go sdk v2 version to 1.24.0.
  • The best explanation I could find for what happened was in https://github.com/eksctl-io/eksctl/pull/7487. eksctl resolved it by removing the RetryMetricsHeader from the middleware, but then later reverted that change after upgrading to aws go sdk v2 version 1.25.0.
  • The new and improved ClusterAuth uptest passes on this branch, but I'd still appreciate
  • [ ] Someone with more context on the issue validating that it's not recurring
• [x] EMR Serverless Application type change
  • https://github.com/aws/aws-sdk-go-v2/pull/2458 introduced a breakfix change to emrserverless, changing a type, which causes a compilation error if we upgrade just the aws sdk without also consuming the corresponding fix from the terraform aws provider.
  • Once consuming both pieces, uptest passes :heavy_check_mark: for the emrserverless application

Potentially incompatible schema changes:

From https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8792472872/job/24128687844

Report breaking CRD OpenAPI v3 schema changes:

• [x] package/crds/cloud9.aws.upbound.io_environmentec2s.yaml
  • Uptest successful
• [x] package/crds/dms.aws.upbound.io_eventsubscriptions.yaml
  • This is not currently uptestable, and looks not trivial to make uptestable. The change is marking a field which was erroneously made required in TF provider 5.31 back to optional. Based on looking at the change, I don't think this will be a problem.
• [x] package/crds/fsx.aws.upbound.io_ontapfilesystems.yaml
  • Uptest successful

Report native schema version changes

• [x] aws_secretsmanager_secret_rotation
  • IUptest successful

Uptest runs

:heavy_check_mark: EKS ClusterAuth: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8794111828 :heavy_check_mark: FSX OnTapFileSystem: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8794383020 :heavy_check_mark: EMR Serverless Application: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8794565770 :heavy_check_mark: Cloud9 EnvironmentEc2: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8795118229 :heavy_check_mark: Secretsmanager SecretRotation: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8795106214

Terraform provider changelog lines to look into:

This is a very light filter through the tf provider changelog, with things I though might be slightly notable or worth checking. I'm sure there are a bunch of things on here that just won't matter at all.

• [ ] 5.42.0 BUG FIXES * resource/aws_appautoscaling_scheduled_action: Read correct resource by using scalable_dimension as an additional filter
• [ ] 5.37.0 BUG FIXES * resource/aws_autoscaling_group: Fix version to computed for inconsistent final plan issue
• [ ] 5.44.0 ENHANCEMENTS * resource/aws_batch_job_definition: Add update functions instead of ForceNew. Add deregister_on_new_revision to allow keeping prior versions ACTIVE when a new revision is published.
• [ ] 5.46.0 BUG FIXES * resource/aws_ce_anomaly_monitor: Change monitor_dimension to [ForceNew]
• [x] 5.32.0 BUG FIXES * resource/aws_cloud9_environment_ec2: image_id is Required
• [ ] 5.46.0 ENHANCEMENTS * resource/aws_cloudformation_stack_set: Add retry when creating to potentially help with eventual consistency problems
• [ ] 5.36.0 ENHANCEMENTS * resource/aws_cloudwatch_metric_stream: Add plan-time validation of output_format
• [ ] 5.34.0 ENHANCEMENTS * resource/aws_codecommit_repository: Add kms_key_id argument
  • (This one is causing issues with the codegen pipeline)
• [ ] 5.34.0 ENHANCEMENTS * resource/aws_codecommit_trigger: Add plan-time validation of trigger.destination_arn and trigger.events
• [ ] 5.39.0 ENHANCEMENTS * resource/aws_config_configuration_recorder: Add plan-time validation of aws_config_organization_custom_rule.lambda_function_arn
• [ ] 5.42.0 BUG FIXES * resource/aws_datasync_location_s3: Fix missing s3_bucket_arn attribute value from state read/refresh
• [ ] 5.38.0 ENHANCEMENTS * resource/aws_db_instance: Add plan-time validation of performance_insights_retention_period
• [ ] 5.40.0 BUG FIXES * resource/aws_db_instance: Correctly sets parameter_group_name when replicate_source_db is in different region.
• [ ] 5.44.0 ENHANCEMENTS * resource/aws_db_instance: Adds warning when setting character_set_name when replicate_source_db, restore_to_point_in_time, or snapshot_identifier is set
• [ ] 5.37.0 BUG FIXES * resource/aws_db_proxy: Change auth from TypeList to TypeSet as order is not significant
• [ ] 5.35.0 BUG FIXES * resource/aws_dms_event_subscription: Mark source_ids as Optional. This fixes a regression introduced in [v5.31.0]
• [ ] 5.46.0 BUG FIXES * resource/aws_dms_replication_task: Allows leaving replication_task_settings unset to use default settings.
• [ ] 5.41.0 ENHANCEMENTS * resource/aws_ecs_cluster: Add default value (DEFAULT) for configuration.execute_command_configuration.logging
• [ ] 5.41.0 BUG FIXES * resource/aws_ecs_cluster: Fix panic: interface conversion: interface {} is nil, not map[string]interface {} when configuration, configuration.execute_command_configuration, or configuration.execute_command_configuration.log_configuration are empty
• [ ] 5.41.0 BUG FIXES * resource/aws_ecs_service: service_connect_configuration.service.tls.issuer_cert_authority.aws_pca_authority_arn is Required
• [ ] 5.35.0 BUG FIXES * resource/aws_efs_file_system: Increase lifecycle_policy maximum item limit to 3
• [ ] 5.32.0 ENHANCEMENTS * resource/aws_fsx_ontap_file_system: throughput_capacity is Optional
• [ ] 5.46.0 BUG FIXES * resource/aws_glue_job: Mark number_of_workers and worker_type as optional/computed, preventing persistent differences when max_capacity is set.
• [ ] 5.39.0 ENHANCEMENTS resource/aws_instance: Apply default tags to volumes/block devices managed through an aws_instance, add `ebs_block_device..tags_allandroot_block_device.*.tags_all` attributes which include default tags
• [ ] 5.36.0 BUG FIXES * resource/aws_kinesis_firehose_delivery_stream: Change extended_s3_configuration.processing_configuration.processors.parameters from TypeList to TypeSet as order is not significant
• [ ] 5.32.0 BUG FIXES * resource/aws_lambda_function: Ensure lambda does not get deployed if source_code_hash does not change.
• [ ] 5.38.0 ENHANCEMENTS * resource/aws_lb_target_group: Add load_balancer_arns attribute
• [ ] 5.36.0 NOTES * resource/aws_media_convert_queue: The AWS Elemental MediaConvert service has been converted to use standard Regional endpoints instead of deprecated per-account endpoints
• [ ] 5.41.0 BUG FIXES * resource/aws_network_acl_rule: Fix InvalidNetworkAclID.NotFound errors on resource Delete
• [ ] 5.41.0 BUG FIXES * resource/aws_network_acl_rule: Prevent creation of duplicate Terraform resources
• [ ] 5.41.0 BUG FIXES * resource/aws_ram_principal_association: Prevent creation of duplicate Terraform resources
• [ ] 5.41.0 BUG FIXES * resource/aws_ram_resource_association: Prevent creation of duplicate Terraform resources
• [ ] 5.45.0 NOTES * resource/aws_redshift_cluster: The logging argument is now deprecated. Use the aws_redshift_logging resource instead.
• [ ] 5.45.0 NOTES * resource/aws_redshift_cluster: The snapshot_copy argument is now deprecated. Use the aws_redshift_snapshot_copy resource instead.
• [ ] 5.41.0 BUG FIXES * resource/aws_route: Prevent creation of duplicate Terraform resources
• [ ] 5.43.0 BUG FIXES * resource/aws_route53_zone: Prevent re-creation when name casing changes
• [ ] 5.32.0 ENHANCEMENTS * resource/aws_s3_bucket: Modify resource Read to support third-party S3 API implementations. Because we cannot easily test this functionality, it is best effort and we ask for community help in testing
• [ ] 5.32.0 BUG FIXES * resource/aws_s3_bucket_object: Remove any leading ./ from key to maintain AWS SDK for Go v1 (pre-v5.17.0) compatibility
• [ ] 5.32.0 BUG FIXES * resource/aws_s3_object: Remove any leading ./ from key to maintain AWS SDK for Go v1 (pre-v5.17.0) compatibility
• [ ] 5.32.0 BUG FIXES * resource/aws_s3_object_copy: Remove any leading ./ from key to maintain AWS SDK for Go v1 (pre-v5.17.0) compatibility
• [ ] 5.33.0 BUG FIXES * resource/aws_secretsmanager_secret_version: Fix InvalidParameterException: The parameter RemoveFromVersionId can't be empty. Staging label AWSCURRENT is currently attached to version ..., so you must explicitly reference that version in RemoveFromVersionId errors when a secret is updated outside Terraform
• [ ] 5.43.0 BUG FIXES * resource/aws_secretsmanager_secret_version: Fix to handle versions deleted out-of-band without raising an InvalidRequestException
• [ ] 5.43.0 BUG FIXES * resource/aws_ssm_parameter: force create a new SSM parameter when data_type is updated.

[mbbush]

/test-examples="examples/eks/v1beta1/clusterauth.yaml"

[mbbush]

/test-examples="examples/fsx/v1beta1/ontapfilesystem.yaml"

[mbbush]

/test-examples="examples/emrserverless/v1beta1/application.yaml"

[mbbush]

/test-examples="examples/secretsmanager/v1beta1/secretrotation.yaml"

[mbbush]

/test-examples="examples/cloud9/v1beta1/environmentec2.yaml"

[mbbush]

/test-examples="examples/ecs/v1beta1/cluster.yaml"

[mbbush]

/test-examples="examples/appautoscaling/v1beta1/scheduledaction.yaml"

[johnathan-sq]

Any updates on this PR? Happy to help out where I can.

[jeanduplessis]

@johnathan-sq we will be focusing on finalizing the TF provider upgrade next week.

[turkenf]

Superseded by https://github.com/crossplane-contrib/provider-upjet-aws/pull/1315