tests/provider: Add concurrent sweepers

YakDriver commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When running a test suite, sweepers run before to make sure the environment is clean. However, these sweepers can take a very long time to run. Adding concurrency, especially to long-running sweepers, is likely to improve sweeper performance.

Documentation

This is the documentation for writing and/or updating sweepers in the tests. Also, refer to the PRs below for examples.

Top 10 Longest Tests

[x] resource_aws_waf_web_acl_test.go (#15359)
[x] resource_aws_elasticsearch_domain_test.go (#18574)
[x] resource_aws_eks_node_group_test.go (#18575)
[x] resource_aws_redshift_cluster_test.go (#18576)
[x] resource_aws_elasticache_replication_group_test.go (#18587)
[x] resource_aws_redshift_snapshot_schedule_test.go (#18592)
[x] resource_aws_db_instance_test.go (#15336)
[ ] resource_aws_dms_replication_instance_test.go (#18623)
[ ] resource_aws_fsx_windows_file_system_test.go (#18626)
[ ] resource_aws_waf_rule_group_test.go (#18631)

6 Sweepers Running the Longest, April 7, 2021

[ ] aws_instance (#18652)
[x] aws_elasticache_replication_group (#18587)
[ ] aws_subnet (#18653)
[ ] aws_ecrpublic_repository (#18656)
[ ] aws_security_group (#18657)
[ ] aws_vpc (#18658)

Affected Resources

[ ] resource_aws_acm_certificate_test.go
[ ] resource_aws_acmpca_certificate_authority_test.go
[ ] resource_aws_api_gateway_rest_api_test.go
[ ] resource_aws_api_gateway_vpc_link_test.go
[ ] resource_aws_apigatewayv2_api_test.go
[ ] resource_aws_apigatewayv2_domain_name_test.go
[ ] resource_aws_apigatewayv2_vpc_link_test.go
[ ] resource_aws_appmesh_mesh_test.go
[ ] resource_aws_appmesh_route_test.go
[ ] resource_aws_appmesh_virtual_node_test.go
[ ] resource_aws_appmesh_virtual_router_test.go
[ ] resource_aws_appmesh_virtual_service_test.go
[ ] resource_aws_appsync_graphql_api_test.go
[ ] resource_aws_autoscaling_group_test.go
[ ] resource_aws_batch_compute_environment_test.go
[ ] resource_aws_batch_job_definition_test.go
[ ] resource_aws_batch_job_queue_test.go
[ ] resource_aws_budgets_budget_test.go
[ ] resource_aws_cloudformation_stack_set_instance_test.go
[ ] resource_aws_cloudformation_stack_set_test.go
[ ] resource_aws_cloudformation_stack_test.go
[ ] resource_aws_cloudfront_distribution_test.go
[ ] resource_aws_cloudhsm2_cluster_test.go
[ ] resource_aws_cloudtrail_test.go
[ ] resource_aws_cloudwatch_event_permission_test.go
[ ] resource_aws_cloudwatch_event_rule_test.go
[ ] resource_aws_cloudwatch_event_target_test.go
[ ] resource_aws_cloudwatch_log_group_test.go
[ ] resource_aws_cloudwatch_log_resource_policy_test.go
[ ] resource_aws_cognito_user_pool_domain_test.go
[ ] resource_aws_cognito_user_pool_test.go
[ ] resource_aws_config_aggregate_authorization_test.go
[ ] resource_aws_config_configuration_aggregator_test.go
[ ] resource_aws_config_configuration_recorder_test.go
[ ] resource_aws_config_delivery_channel_test.go
[ ] resource_aws_datasync_agent_test.go
[ ] resource_aws_datasync_location_efs_test.go
[ ] resource_aws_datasync_location_fsx_windows_file_system_test.go
[ ] resource_aws_datasync_location_nfs_test.go
[ ] resource_aws_datasync_location_s3_test.go
[ ] resource_aws_datasync_location_smb_test.go
[ ] resource_aws_datasync_task_test.go
[ ] resource_aws_dax_cluster_test.go
[ ] resource_aws_db_cluster_snapshot_test.go
[ ] resource_aws_db_event_subscription_test.go
[ ] resource_aws_db_option_group_test.go
[ ] resource_aws_db_parameter_group_test.go
[ ] resource_aws_db_proxy_test.go
[ ] resource_aws_db_snapshot_test.go
[ ] resource_aws_db_subnet_group_test.go
[ ] resource_aws_directory_service_directory_test.go
[ ] resource_aws_dx_gateway_association_proposal_test.go
[ ] resource_aws_dx_gateway_association_test.go
[ ] resource_aws_dx_gateway_test.go
[ ] resource_aws_dynamodb_table_test.go
[ ] resource_aws_ebs_volume_test.go
[ ] resource_aws_ec2_capacity_reservation_test.go
[ ] resource_aws_ec2_client_vpn_endpoint_test.go
[ ] resource_aws_ec2_client_vpn_network_association_test.go
[ ] resource_aws_ec2_transit_gateway_peering_attachment_test.go
[ ] resource_aws_ec2_transit_gateway_test.go
[ ] resource_aws_ec2_transit_gateway_vpc_attachment_test.go
[ ] resource_aws_ecr_repository_test.go
[ ] resource_aws_ecs_capacity_provider_test.go
[ ] resource_aws_ecs_cluster_test.go
[ ] resource_aws_ecs_service_test.go
[ ] resource_aws_ecs_task_definition_test.go
[ ] resource_aws_efs_access_point_test.go
[ ] resource_aws_efs_file_system_test.go
[ ] resource_aws_efs_mount_target_test.go
[ ] resource_aws_egress_only_internet_gateway_test.go
[ ] resource_aws_eip_test.go
[ ] resource_aws_eks_cluster_test.go
[ ] resource_aws_eks_fargate_profile_test.go
[ ] resource_aws_elastic_beanstalk_application_test.go
[ ] resource_aws_elastic_beanstalk_environment_test.go
[ ] resource_aws_elasticache_cluster_test.go
[ ] resource_aws_elasticache_security_group_test.go
[ ] resource_aws_elb_test.go
[ ] resource_aws_emr_cluster_test.go
[ ] resource_aws_flow_log_test.go
[ ] resource_aws_fsx_lustre_file_system_test.go
[ ] resource_aws_gamelift_alias_test.go
[ ] resource_aws_gamelift_build_test.go
[ ] resource_aws_gamelift_fleet_test.go
[ ] resource_aws_gamelift_game_session_queue_test.go
[ ] resource_aws_glacier_vault_test.go
[ ] resource_aws_globalaccelerator_accelerator_test.go
[ ] resource_aws_glue_classifier_test.go
[ ] resource_aws_glue_connection_test.go
[ ] resource_aws_glue_crawler_test.go
[ ] resource_aws_glue_job_test.go
[ ] resource_aws_glue_security_configuration_test.go
[ ] resource_aws_glue_trigger_test.go
[ ] resource_aws_glue_workflow_test.go
[ ] resource_aws_guardduty_detector_test.go
[ ] resource_aws_guardduty_publishing_destination_test.go
[ ] resource_aws_iam_group_test.go
[ ] resource_aws_iam_policy_test.go
[ ] resource_aws_iam_role_test.go
[ ] resource_aws_iam_server_certificate_test.go
[ ] resource_aws_iam_service_linked_role_test.go
[ ] resource_aws_iam_user_test.go
[ ] resource_aws_instance_test.go
[ ] resource_aws_internet_gateway_test.go
[ ] resource_aws_iot_topic_rule_test.go
[ ] resource_aws_key_pair_test.go
[ ] resource_aws_kinesis_analytics_application_test.go
[ ] resource_aws_kinesis_firehose_delivery_stream_test.go
[ ] resource_aws_kinesis_stream_test.go
[ ] resource_aws_kms_key_test.go
[ ] resource_aws_lambda_function_test.go
[ ] resource_aws_lambda_layer_version_test.go
[ ] resource_aws_launch_configuration_test.go
[ ] resource_aws_launch_template_test.go
[ ] resource_aws_lb_target_group_test.go
[ ] resource_aws_lb_test.go
[ ] resource_aws_licensemanager_license_configuration_test.go
[ ] resource_aws_lightsail_instance_test.go
[ ] resource_aws_lightsail_static_ip_test.go
[ ] resource_aws_mq_broker_test.go
[ ] resource_aws_msk_cluster_test.go
[ ] resource_aws_msk_configuration_test.go
[ ] resource_aws_nat_gateway_test.go
[ ] resource_aws_neptune_event_subscription_test.go
[ ] resource_aws_network_acl_test.go
[ ] resource_aws_network_interface_test.go
[ ] resource_aws_pinpoint_app_test.go
[ ] resource_aws_qldb_ledger_test.go
[ ] resource_aws_rds_cluster_parameter_group_test.go
[ ] resource_aws_rds_cluster_test.go
[ ] resource_aws_rds_global_cluster_test.go
[ ] resource_aws_redshift_cluster_snapshot_test.go
[ ] resource_aws_redshift_event_subscription_test.go
[ ] resource_aws_redshift_subnet_group_test.go
[ ] resource_aws_route_table_test.go
[ ] resource_aws_route53_query_log_test.go
[ ] resource_aws_route53_resolver_endpoint_test.go
[ ] resource_aws_route53_resolver_query_log_config_association_test.go
[ ] resource_aws_route53_resolver_query_log_config_test.go
[ ] resource_aws_route53_resolver_rule_association_test.go
[ ] resource_aws_route53_resolver_rule_test.go
[ ] resource_aws_s3_access_point_test.go
[ ] resource_aws_s3_bucket_object_test.go
[ ] resource_aws_s3_bucket_test.go
[ ] resource_aws_sagemaker_endpoint_configuration_test.go
[ ] resource_aws_sagemaker_endpoint_test.go
[ ] resource_aws_sagemaker_model_test.go
[ ] resource_aws_sagemaker_notebook_instance_lifecycle_configuration_test.go
[ ] resource_aws_sagemaker_notebook_instance_test.go
[ ] resource_aws_secretsmanager_secret_test.go
[ ] resource_aws_security_group_test.go
[ ] resource_aws_service_discovery_http_namespace_test.go
[ ] resource_aws_service_discovery_private_dns_namespace_test.go
[ ] resource_aws_service_discovery_public_dns_namespace_test.go
[ ] resource_aws_service_discovery_service_test.go
[ ] resource_aws_ses_configuration_set_test.go
[ ] resource_aws_ses_domain_identity_test.go
[ ] resource_aws_ses_receipt_rule_set_test.go
[ ] resource_aws_sns_platform_application_test.go
[ ] resource_aws_sns_topic_test.go
[ ] resource_aws_spot_fleet_request_test.go
[ ] resource_aws_sqs_queue_test.go
[ ] resource_aws_ssm_maintenance_window_test.go
[ ] resource_aws_storagegateway_gateway_test.go
[ ] resource_aws_subnet_test.go
[ ] resource_aws_transfer_server_test.go
[ ] resource_aws_vpc_dhcp_options_test.go
[ ] resource_aws_vpc_endpoint_service_test.go
[ ] resource_aws_vpc_endpoint_test.go
[ ] resource_aws_vpc_peering_connection_test.go
[ ] resource_aws_vpc_test.go
[ ] resource_aws_vpn_connection_test.go
[ ] resource_aws_vpn_gateway_test.go
[ ] resource_aws_waf_regex_match_set_test.go
[ ] resource_aws_wafregional_rate_based_rule_test.go
[ ] resource_aws_wafregional_regex_match_set_test.go
[ ] resource_aws_wafregional_rule_group_test.go
[ ] resource_aws_wafregional_rule_test.go
[ ] resource_aws_wafregional_web_acl_test.go
[ ] resource_aws_workspaces_directory_test.go
[ ] resource_aws_workspaces_workspace_test.go

References

Acceptance Test Sweepers

YakDriver commented 4 years ago

To sweep an individual resource on GovCloud:

SWEEP=us-gov-west-1 SWEEPARGS=-sweep-run=aws_db_instance make sweep

Tensho commented 3 years ago

@YakDriver I wonder, doesn't sweepers runner call dependencies sweep function in parallel? Just a note, bothaws_workspaces_directory and aws_workspaces_workspace resources take a significant amount of time to be destroyed even for one instance case.

YakDriver commented 3 years ago

@YakDriver I wonder, doesn't sweepers runner call dependencies sweep function in parallel?

Unfortunately, not currently.

github-actions[bot] commented 8 months ago

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

github-actions[bot] commented 6 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws