Add Metric Collection similar to moto tests

[baermat]

This PR adds metric collection to the terraform tests like we have them for the moto tests we run against LocalStack. This includes the following changes:

• The default image to test against is now localstack/localstack-pro
• All applicable services are tested, not just community services anymore
• LocalStack is now started via localstack cli command in the pipeline
• Services that take too long are divided into partitions to avoid timeouts. Right now this includes ecs, ec2, and elasticache. This service-partition list is not created automatically as of this point. All other service have one partition named 'All'
• Removed the test parallelization for ec2. Doubled the number of initial partitions in response
• All metrics are collected in csv files and uploaded in the final artifacts. One file per partition is created
• Replace the option to specify arbitrary images with a switch whether to enable the pro image or not (in which case community is used)

[github-actions[bot]]

sqs Terraform Test Results

0 tests   - 35   0 :heavy_check_mark:  - 26   0s :stopwatch: - 54m 20s 0 suites  -   1   0 :zzz: ±  0  0 files    -   1   0 :x:  -   9 

Results for commit 2a1e033a. ± Comparison against base commit 094ed90e.

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

sns Terraform Test Results

0 tests   - 38   0 :heavy_check_mark:  - 30   0s :stopwatch: - 11m 21s 0 suites  -   1   0 :zzz: ±  0  0 files    -   1   0 :x:  -   8 

Results for commit 2a1e033a. ± Comparison against base commit 094ed90e.

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

iotanalytics Terraform Test Results

0 tests   0 :heavy_check_mark:  0s :stopwatch: 0 suites  0 :zzz: 0 files    0 :x:

Results for commit 9c1a723a.

[github-actions[bot]]

fis Terraform Test Results

3 tests   1 :heavy_check_mark:  1m 3s :stopwatch: 1 suites  0 :zzz: 1 files    2 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

appconfig Terraform Test Results

30 tests     0 :heavy_check_mark:  3m 28s :stopwatch:   1 suites    0 :zzz:   1 files    30 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

cloudtrail Terraform Test Results

8 tests   2 :heavy_check_mark:  4m 23s :stopwatch: 1 suites  0 :zzz: 1 files    6 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

ecr Terraform Test Results

25 tests   16 :heavy_check_mark:  5m 20s :stopwatch:   1 suites    0 :zzz:   1 files      9 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

kafka Terraform Test Results

41 tests     0 :heavy_check_mark:  4m 13s :stopwatch:   1 suites    0 :zzz:   1 files    41 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

s3control Terraform Test Results

44 tests  ±0   11 :heavy_check_mark: ±0   7m 7s :stopwatch: - 2m 46s   1 suites ±0     0 :zzz: ±0    1 files   ±0   33 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

cloudwatch Terraform Test Results

33 tests  ±0   20 :heavy_check_mark: ±0   9m 3s :stopwatch: - 3m 42s   1 suites ±0     0 :zzz: ±0    1 files   ±0   13 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

transcribe Terraform Test Results

17 tests  ±0     0 :heavy_check_mark: ±0   1m 25s :stopwatch: - 1m 13s   1 suites ±0     0 :zzz: ±0    1 files   ±0   17 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

elasticbeanstalk Terraform Test Results

24 tests     2 :heavy_check_mark:  2m 4s :stopwatch:   1 suites    0 :zzz:   1 files    22 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

sts Terraform Test Results

1 tests  ±0   1 :heavy_check_mark: ±0   41s :stopwatch: -36s 1 suites ±0   0 :zzz: ±0  1 files   ±0   0 :x: ±0 

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

cloudformation Terraform Test Results

48 tests  ±0   12 :heavy_check_mark: +2   16m 57s :stopwatch: +16s   1 suites ±0     0 :zzz: ±0    1 files   ±0   36 :x:  - 2 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

iot Terraform Test Results

67 tests   10 :heavy_check_mark:  11m 28s :stopwatch:   1 suites    0 :zzz:   1 files    57 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

batch Terraform Test Results

45 tests   18 :heavy_check_mark:  15m 5s :stopwatch:   1 suites    0 :zzz:   1 files    27 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

organizations Terraform Test Results

8 tests   0 :heavy_check_mark:  2m 58s :stopwatch: 1 suites  0 :zzz: 1 files    8 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

cognitoidentity Terraform Test Results

18 tests   9 :heavy_check_mark:  4m 8s :stopwatch:   1 suites  0 :zzz:   1 files    9 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

athena Terraform Test Results

36 tests     1 :heavy_check_mark:  1h 5m 58s :stopwatch:   1 suites    0 :zzz:   1 files    35 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

acm Terraform Test Results

40 tests  ±0   29 :heavy_check_mark: ±0   55m 7s :stopwatch: - 3m 26s   1 suites ±0     0 :zzz: ±0    1 files   ±0   11 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

redshift Terraform Test Results

83 tests  ±0     6 :heavy_check_mark: +1   12m 21s :stopwatch: + 3m 49s   1 suites ±0     0 :zzz: ±0    1 files   ±0   77 :x:  - 1 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

glue Terraform Test Results

168 tests     64 :heavy_check_mark:  1h 32m 17s :stopwatch:     1 suites      0 :zzz:     1 files    104 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

amplify Terraform Test Results

1 tests   0 :heavy_check_mark:  8m 24s :stopwatch: 1 suites  0 :zzz: 1 files    1 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

route53 Terraform Test Results

99 tests  ±0   65 :heavy_check_mark: +1   1h 23m 3s :stopwatch: + 1m 38s   1 suites ±0     0 :zzz: ±0    1 files   ±0   34 :x:  - 1 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

logs Terraform Test Results

38 tests  ±0   19 :heavy_check_mark: ±0   9m 26s :stopwatch: -9s   1 suites ±0     0 :zzz: ±0    1 files   ±0   19 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

neptune Terraform Test Results

50 tests     7 :heavy_check_mark:  1h 38m 2s :stopwatch:   1 suites    0 :zzz:   1 files    43 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

docdb Terraform Test Results

57 tests     4 :heavy_check_mark:  1h 56m 49s :stopwatch:   1 suites    0 :zzz:   1 files    53 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

kms Terraform Test Results

67 tests  ±0   40 :heavy_check_mark: ±0   27m 4s :stopwatch: - 1m 28s   1 suites ±0     0 :zzz: ±0    1 files   ±0   27 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

backup Terraform Test Results

45 tests     0 :heavy_check_mark:  3m 57s :stopwatch:   1 suites    0 :zzz:   1 files    45 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

mediastore Terraform Test Results

3 tests   0 :heavy_check_mark:  2h 32m 27s :stopwatch: 1 suites  0 :zzz: 1 files    3 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

kinesis Terraform Test Results

22 tests  ±0   20 :heavy_check_mark: ±0   1h 11m 7s :stopwatch: -27s   1 suites ±0     0 :zzz: ±0    1 files   ±0     2 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

iam Terraform Test Results

176 tests  ±0   129 :heavy_check_mark: ±0   3h 15m 46s :stopwatch: + 2h 34m 26s     1 suites ±0       0 :zzz: ±0      1 files   ±0     47 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

cognitoidp Terraform Test Results

85 tests   51 :heavy_check_mark:  1h 9m 38s :stopwatch:   1 suites    0 :zzz:   1 files    34 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

efs Terraform Test Results

44 tests     8 :heavy_check_mark:  11m 8s :stopwatch:   1 suites    0 :zzz:   1 files    36 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

secretsmanager Terraform Test Results

29 tests  ±0   19 :heavy_check_mark: ±0   7m 31s :stopwatch: -4s   1 suites ±0     0 :zzz: ±0    1 files   ±0   10 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

dynamodb Terraform Test Results

60 tests  ±0   41 :heavy_check_mark: ±0   28m 32s :stopwatch: -26s   1 suites ±0     0 :zzz: ±0    1 files   ±0   19 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

eks Terraform Test Results

72 tests     7 :heavy_check_mark:  1h 58m 6s :stopwatch:   1 suites    0 :zzz:   1 files    65 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

lakeformation Terraform Test Results

7 tests   3 :heavy_check_mark:  20m 5s :stopwatch: 1 suites  0 :zzz: 1 files    4 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

resourcegroupstaggingapi Terraform Test Results

4 tests  ±0   3 :heavy_check_mark: ±0   1m 3s :stopwatch: -28s 1 suites ±0   0 :zzz: ±0  1 files   ±0   1 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

autoscaling Terraform Test Results

126 tests     20 :heavy_check_mark:  1h 29m 37s :stopwatch:     1 suites      0 :zzz:     1 files    106 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

elasticsearch Terraform Test Results

46 tests  ±0   38 :heavy_check_mark: +2   3h 0m 40s :stopwatch: -33s   1 suites ±0     0 :zzz: ±0    1 files   ±0     8 :x:  - 2 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

cloudfront Terraform Test Results

93 tests   19 :heavy_check_mark:  41m 44s :stopwatch:   1 suites    0 :zzz:   1 files    74 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

resourcegroups Terraform Test Results

2 tests  ±0   0 :heavy_check_mark: ±0   1h 40m 5s :stopwatch: + 1h 38m 2s 1 suites ±0   0 :zzz: ±0  1 files   ±0   2 :x: ±0 

For more details on these failures, see this check.

Results for commit 9c1a723a. ± Comparison against base commit 094ed90e.

[github-actions[bot]]

codecommit Terraform Test Results

15 tests     2 :heavy_check_mark:  2m 49s :stopwatch:   1 suites    0 :zzz:   1 files    13 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

emr Terraform Test Results

0 tests   0 :heavy_check_mark:  0s :stopwatch: 0 suites  0 :zzz: 0 files    0 :x:

Results for commit 9c1a723a.

[github-actions[bot]]

qldb Terraform Test Results

0 tests   0 :heavy_check_mark:  0s :stopwatch: 0 suites  0 :zzz: 0 files    0 :x:

Results for commit 9c1a723a.

[github-actions[bot]]

sagemaker Terraform Test Results

0 tests   0 :heavy_check_mark:  0s :stopwatch: 0 suites  0 :zzz: 0 files    0 :x:

Results for commit 9c1a723a.

[github-actions[bot]]

elb Terraform Test Results

44 tests   20 :heavy_check_mark:  8m 36s :stopwatch:   1 suites    0 :zzz:   1 files    24 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

appsync Terraform Test Results

3 tests   0 :heavy_check_mark:  2h 17m 24s :stopwatch: 1 suites  0 :zzz: 1 files    3 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.

[github-actions[bot]]

apigatewayv2 Terraform Test Results

83 tests   44 :heavy_check_mark:  2h 52m 34s :stopwatch:   1 suites    0 :zzz:   1 files    39 :x:

For more details on these failures, see this check.

Results for commit 9c1a723a.