cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.21k stars 3.82k forks source link

roachtest: cluster_creation failed #105298

Closed cockroach-teamcity closed 1 year ago

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 1eb628e8c8a7eb1cbf9bfa2bd6c31982c25cbba0:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,arch=amd64,created=2023-06-21t17_48_41z,roachprod=true,cluster=teamcity-10611064-1687326486-148-n9cpu8-geo,lifetime=12h0m0s --metadata-from-file startup-script=/tmp/gce-startup-script2637264012 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10611064-1687326486-148-n9cpu8-geo-0004 teamcity-10611064-1687326486-148-n9cpu8-geo-0005 teamcity-10611064-1687326486-148-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

/cc @cockroachdb/dev-inf

This test on roachdash | Improve this report!

Jira issue: CRDB-28972

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=32/size=64kb was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:46:30Z},{Key=Name,Value=teamcity-10644057-1687499112-03-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-03-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:46:30Z},{Key=Name,Value=teamcity-10644057-1687499112-03-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-03-n4cpu32}] --user-data file:///tmp/aws-startup-script2360427841 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2690434951 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=32/splt=0 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-23T05:48:08Z},{Key=Name,Value=teamcity-10644057-1687499112-06-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-06-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-23T05:48:08Z},{Key=Name,Value=teamcity-10644057-1687499112-06-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-06-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script4093103990 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2881340379 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=32/size=4kb was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-10-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:51:03Z},{Key=Name,Value=teamcity-10644057-1687499112-10-n4cpu32-0004}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-10-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:51:03Z},{Key=Name,Value=teamcity-10644057-1687499112-10-n4cpu32-0004}] --user-data file:///tmp/aws-startup-script3823569849 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2237168753 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-13-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:52:41Z},{Key=Name,Value=teamcity-10644057-1687499112-13-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-13-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:52:41Z},{Key=Name,Value=teamcity-10644057-1687499112-13-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script50036950 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1759803903 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-16-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:54:21Z},{Key=Name,Value=teamcity-10644057-1687499112-16-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-16-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T05:54:21Z},{Key=Name,Value=teamcity-10644057-1687499112-16-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script4058082048 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping200857282 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=1/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-23T05:58:29Z},{Key=Name,Value=teamcity-10644057-1687499112-21-n2cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-21-n2cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-23T05:58:29Z},{Key=Name,Value=teamcity-10644057-1687499112-21-n2cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-21-n2cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script585349922 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping199555758 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=32/seq was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-24-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:00:07Z},{Key=Name,Value=teamcity-10644057-1687499112-24-n4cpu32-0004}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-24-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:00:07Z},{Key=Name,Value=teamcity-10644057-1687499112-24-n4cpu32-0004}] --user-data file:///tmp/aws-startup-script89582096 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping798827612 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv0/enc=false/nodes=3/cpu=96 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.24xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-23T06:12:35Z},{Key=Name,Value=teamcity-10644057-1687499112-36-n4cpu96-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-36-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-23T06:12:35Z},{Key=Name,Value=teamcity-10644057-1687499112-36-n4cpu96-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-36-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script2980719147 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3969216796 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.24xlarge capacity in the Availability Zone you requested (us-east-2b). Our system will be working on provisioning additional capacity. You can currently get c6id.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2c.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=96 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv95/enc=false/nodes=3/cpu=32/seq was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-10644057-1687499112-46-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:47:58Z},{Key=Name,Value=teamcity-10644057-1687499112-46-n4cpu32-0003},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-10644057-1687499112-46-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:47:58Z},{Key=Name,Value=teamcity-10644057-1687499112-46-n4cpu32-0003},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3419050897 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1560523593 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv95/enc=false/nodes=3/cpu=32/size=4kb was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-10644057-1687499112-49-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:49:32Z},{Key=Name,Value=teamcity-10644057-1687499112-49-n4cpu32-0002},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-10644057-1687499112-49-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:49:32Z},{Key=Name,Value=teamcity-10644057-1687499112-49-n4cpu32-0002},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3267014812 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2827241542 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test kv95/enc=false/nodes=3/cpu=32/size=64kb was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:51:12Z},{Key=Name,Value=teamcity-10644057-1687499112-52-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-52-n4cpu32},{Key=Lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T06:51:12Z},{Key=Name,Value=teamcity-10644057-1687499112-52-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-52-n4cpu32},{Key=Lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script3483839754 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping591344616 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/B/nodes=3/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-67-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:37:42Z},{Key=Name,Value=teamcity-10644057-1687499112-67-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-67-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:37:42Z},{Key=Name,Value=teamcity-10644057-1687499112-67-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script1996864015 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1770597261 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/B/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-70-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:39:13Z},{Key=Name,Value=teamcity-10644057-1687499112-70-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-70-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:39:13Z},{Key=Name,Value=teamcity-10644057-1687499112-70-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script1149595262 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping870508395 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/C/nodes=3/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-73-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:40:45Z},{Key=Name,Value=teamcity-10644057-1687499112-73-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-73-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:40:45Z},{Key=Name,Value=teamcity-10644057-1687499112-73-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script215531154 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3652932845 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/C/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-76-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:42:18Z},{Key=Name,Value=teamcity-10644057-1687499112-76-n4cpu32-0003}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-76-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:42:18Z},{Key=Name,Value=teamcity-10644057-1687499112-76-n4cpu32-0003}] --user-data file:///tmp/aws-startup-script2236605964 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2653769743 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/D/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-79-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:43:54Z},{Key=Name,Value=teamcity-10644057-1687499112-79-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-79-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:43:54Z},{Key=Name,Value=teamcity-10644057-1687499112-79-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script2986318648 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping679118078 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/E/nodes=3/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10644057-1687499112-82-n4cpu32-0003},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-82-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:45:25Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10644057-1687499112-82-n4cpu32-0003},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-82-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:45:25Z}] --user-data file:///tmp/aws-startup-script1534881466 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping9388987 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/E/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-85-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:46:56Z},{Key=Name,Value=teamcity-10644057-1687499112-85-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-85-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:46:56Z},{Key=Name,Value=teamcity-10644057-1687499112-85-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script4167649892 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2845518967 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/F/nodes=3/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10644057-1687499112-88-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-88-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:48:27Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10644057-1687499112-88-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-88-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:48:27Z}] --user-data file:///tmp/aws-startup-script2288843468 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2437477308 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test ycsb/F/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-A4tEvRtggXRyHaDovon_x_HX3-s= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10644057-1687499112-91-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-91-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:49:59Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10644057-1687499112-91-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10644057-1687499112-91-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-23T07:49:59Z}] --user-data file:///tmp/aws-startup-script452809539 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2794785478 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ cb6dd27fdfbfc8a9d0d4753809f4588bae8060d0:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,created=2023-06-23t17_30_20z,roachprod=true,cluster=teamcity-10644056-1687499071-152-n9cpu8-geo,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script1920373351 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10644056-1687499071-152-n9cpu8-geo-0004 teamcity-10644056-1687499071-152-n9cpu8-geo-0005 teamcity-10644056-1687499071-152-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 7fd4c21157221eae9e7d5892d89d2b5a671aba3e:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10657405-1687585562-150-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-06-24t17_12_39z --metadata-from-file startup-script=/tmp/gce-startup-script3401938501 --project cockroach-ephemeral --boot-disk-size=32GB --zone eu-west-1b teamcity-10657405-1687585562-150-n9cpu8-geo-0007 teamcity-10657405-1687585562-150-n9cpu8-geo-0008 teamcity-10657405-1687585562-150-n9cpu8-geo-0009]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/eu-west-1b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ a2c2c060a423ee410b57868e657df644f2619cb3:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10664692-1687671833-157-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-06-25t17_43_52z --metadata-from-file startup-script=/tmp/gce-startup-script4143502050 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10664692-1687671833-157-n9cpu8-geo-0001 teamcity-10664692-1687671833-157-n9cpu8-geo-0002 teamcity-10664692-1687671833-157-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ a2c2c060a423ee410b57868e657df644f2619cb3:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,created=2023-06-26t17_36_03z,roachprod=true,cluster=teamcity-10671650-1687758273-151-n9cpu8-geo,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script3137915077 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10671650-1687758273-151-n9cpu8-geo-0001 teamcity-10671650-1687758273-151-n9cpu8-geo-0002 teamcity-10671650-1687758273-151-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 0b0a212ecbe514b55301f4a963edc92b16de9bab:

test restore/tpce/400GB/aws/nodes=4/cpus=16 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6i.4xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-UnVQgk9E0IOw0zf4e1yZIOK5Lec= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-004c6ad7121a8d5a7 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-27T07:57:56Z},{Key=Name,Value=teamcity-10689033-1687844645-27-n4cpu16-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10689033-1687844645-27-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2023-06-27T07:57:56Z},{Key=Name,Value=teamcity-10689033-1687844645-27-n4cpu16-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10689033-1687844645-27-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script3205996414 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3893324787 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6i.4xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6i.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=aws , ROACHTEST_cpu=16 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 0b0a212ecbe514b55301f4a963edc92b16de9bab:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10689032-1687844707-155-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-06-27t18_01_44z --metadata-from-file startup-script=/tmp/gce-startup-script2481217718 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10689032-1687844707-155-n9cpu8-geo-0001 teamcity-10689032-1687844707-155-n9cpu8-geo-0002 teamcity-10689032-1687844707-155-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ fa47de054aa94a7207f3140cc8dfd87b36f8ec03:

test kv0/enc=false/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-uUNGoeJLp1moczuAOc2RrHq7cw4= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10706495-1687931300-17-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-28T06:05:20Z},{Key=Name,Value=teamcity-10706495-1687931300-17-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10706495-1687931300-17-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-06-28T06:05:20Z},{Key=Name,Value=teamcity-10706495-1687931300-17-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script1989621698 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1156645729 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101290 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot todo-deprecate.branch-release-23.1.0] - #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 4a614f89cea81bf94674d6072c3bbf30502244d4:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,arch=amd64,created=2023-06-29t17_27_04z,roachprod=true,cluster=teamcity-10722908-1688017511-147-n9cpu8-geo,lifetime=12h0m0s --metadata-from-file startup-script=/tmp/gce-startup-script3729758994 --project cockroach-ephemeral --boot-disk-size=32GB --zone eu-west-1b teamcity-10722908-1688017511-147-n9cpu8-geo-0007 teamcity-10722908-1688017511-147-n9cpu8-geo-0008 teamcity-10722908-1688017511-147-n9cpu8-geo-0009]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/eu-west-1b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

blathers-crl[bot] commented 1 year ago

cc @cockroachdb/test-eng

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 84a5961ebde5898d3c97aa97da59719c12a5ed16:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,cluster=teamcity-10735768-1688103921-153-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-06-30t17_42_42z,roachprod=true --metadata-from-file startup-script=/tmp/gce-startup-script1554507858 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10735768-1688103921-153-n9cpu8-geo-0004 teamcity-10735768-1688103921-153-n9cpu8-geo-0005 teamcity-10735768-1688103921-153-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ df82b5306e945d396269f28874d98fa567f4ac32:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10750288-1688190369-151-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-01t18_12_49z --metadata-from-file startup-script=/tmp/gce-startup-script434804038 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10750288-1688190369-151-n9cpu8-geo-0001 teamcity-10750288-1688190369-151-n9cpu8-geo-0002 teamcity-10750288-1688190369-151-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ aacba20d325e5702836e9a76be646b5f1bd922af:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,created=2023-07-02t18_15_55z,roachprod=true,cluster=teamcity-10756585-1688276763-157-n9cpu8-geo,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script2374725665 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10756585-1688276763-157-n9cpu8-geo-0001 teamcity-10756585-1688276763-157-n9cpu8-geo-0002 teamcity-10756585-1688276763-157-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ aacba20d325e5702836e9a76be646b5f1bd922af:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10763018-1688363112-147-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-03t18_00_47z --metadata-from-file startup-script=/tmp/gce-startup-script4255741320 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10763018-1688363112-147-n9cpu8-geo-0004 teamcity-10763018-1688363112-147-n9cpu8-geo-0005 teamcity-10763018-1688363112-147-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 428dc9da6a320de218460de6c6c8807caa4ded98:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10771334-1688449519-162-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-04t18_24_28z --metadata-from-file startup-script=/tmp/gce-startup-script556055204 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10771334-1688449519-162-n9cpu8-geo-0004 teamcity-10771334-1688449519-162-n9cpu8-geo-0005 teamcity-10771334-1688449519-162-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 34699bb9c1557fce449e08a68cd259efec94926f:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10780210-1688536039-154-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-05t18_26_13z --metadata-from-file startup-script=/tmp/gce-startup-script1890981597 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10780210-1688536039-154-n9cpu8-geo-0001 teamcity-10780210-1688536039-154-n9cpu8-geo-0002 teamcity-10780210-1688536039-154-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ dbe8511fae8fca21562fdde5c240b1f7d06ef582:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10797165-1688622302-156-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-06t18_39_12z --metadata-from-file startup-script=/tmp/gce-startup-script915367891 --project cockroach-ephemeral --boot-disk-size=32GB --zone eu-west-1b teamcity-10797165-1688622302-156-n9cpu8-geo-0007 teamcity-10797165-1688622302-156-n9cpu8-geo-0008 teamcity-10797165-1688622302-156-n9cpu8-geo-0009]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/eu-west-1b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 1137aee0089fc2fe161bab0f1c899d2ddc997c90:

test kv0/enc=false/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-SPUNl-3E6Osylh0RCYnr26Rr7cE= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-07T06:00:30Z},{Key=Name,Value=teamcity-10813232-1688708714-15-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10813232-1688708714-15-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-07T06:00:30Z},{Key=Name,Value=teamcity-10813232-1688708714-15-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10813232-1688708714-15-n4cpu32}] --user-data file:///tmp/aws-startup-script1141192465 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping851093745 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 1137aee0089fc2fe161bab0f1c899d2ddc997c90:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,created=2023-07-07t18_22_46z,roachprod=true,cluster=teamcity-10813231-1688708801-151-n9cpu8-geo,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script2441215163 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10813231-1688708801-151-n9cpu8-geo-0001 teamcity-10813231-1688708801-151-n9cpu8-geo-0002 teamcity-10813231-1688708801-151-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 43c26aec0072f76e02e6d5ffc1b7079026b24630:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,arch=amd64,created=2023-07-08t18_19_47z,roachprod=true,cluster=teamcity-10830305-1688795117-155-n9cpu8-geo,lifetime=12h0m0s --metadata-from-file startup-script=/tmp/gce-startup-script2445978592 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10830305-1688795117-155-n9cpu8-geo-0004 teamcity-10830305-1688795117-155-n9cpu8-geo-0005 teamcity-10830305-1688795117-155-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 0207c613fa7c8f3ab66c4518ee1e52dabb863426:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,roachprod=true,cluster=teamcity-10837254-1688881517-152-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-09t18_21_53z --metadata-from-file startup-script=/tmp/gce-startup-script340191180 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10837254-1688881517-152-n9cpu8-geo-0004 teamcity-10837254-1688881517-152-n9cpu8-geo-0005 teamcity-10837254-1688881517-152-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 0207c613fa7c8f3ab66c4518ee1e52dabb863426:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,arch=amd64,created=2023-07-10t18_01_32z,roachprod=true,cluster=teamcity-10843924-1688967924-148-n9cpu8-geo,lifetime=12h0m0s --metadata-from-file startup-script=/tmp/gce-startup-script3718408813 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west-2b teamcity-10843924-1688967924-148-n9cpu8-geo-0004 teamcity-10843924-1688967924-148-n9cpu8-geo-0005 teamcity-10843924-1688967924-148-n9cpu8-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-west-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 382a6efb7985b1993e0398c7b29f0e0b746587f4:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,cluster=teamcity-10858669-1689054288-145-n9cpu8-geo,lifetime=12h0m0s,arch=amd64,created=2023-07-11t18_14_52z,roachprod=true --metadata-from-file startup-script=/tmp/gce-startup-script3472684596 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east-2b teamcity-10858669-1689054288-145-n9cpu8-geo-0001 teamcity-10858669-1689054288-145-n9cpu8-geo-0002 teamcity-10858669-1689054288-145-n9cpu8-geo-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/us-east-2b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ a6dbdee0e07325f58a8baf30c199c9cf7ad46614:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,created=2023-07-12t18_07_35z,roachprod=true,cluster=teamcity-10876386-1689140752-146-n9cpu8-geo,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script1564321061 --project cockroach-ephemeral --boot-disk-size=32GB --zone eu-west-1b teamcity-10876386-1689140752-146-n9cpu8-geo-0007 teamcity-10876386-1689140752-146-n9cpu8-geo-0008 teamcity-10876386-1689140752-146-n9cpu8-geo-0009]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/eu-west-1b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 8dcde9c4635f27cefdbc167729c95bf865d9dfd5:

test kv0/enc=false/nodes=3/cpu=96 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.24xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-CtwW-uCZSG-wD8zkmA-EGjZigTo= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-13T08:47:17Z},{Key=Name,Value=teamcity-10892351-1689227150-37-n4cpu96-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10892351-1689227150-37-n4cpu96}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-13T08:47:17Z},{Key=Name,Value=teamcity-10892351-1689227150-37-n4cpu96-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10892351-1689227150-37-n4cpu96}] --user-data file:///tmp/aws-startup-script441198448 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping590679411 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.24xlarge capacity in the Availability Zone you requested (us-east-2b). Our system will be working on provisioning additional capacity. You can currently get c6id.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2c.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=96 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 8dcde9c4635f27cefdbc167729c95bf865d9dfd5:

test kv95/enc=false/nodes=3/cpu=96 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.24xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-CtwW-uCZSG-wD8zkmA-EGjZigTo= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10892351-1689227150-40-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-13T08:48:31Z},{Key=Name,Value=teamcity-10892351-1689227150-40-n4cpu96-0003}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10892351-1689227150-40-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-13T08:48:31Z},{Key=Name,Value=teamcity-10892351-1689227150-40-n4cpu96-0003}] --user-data file:///tmp/aws-startup-script2961739612 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3516150710 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.24xlarge capacity in the Availability Zone you requested (us-east-2b). Our system will be working on provisioning additional capacity. You can currently get c6id.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2c.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=96 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ 8dcde9c4635f27cefdbc167729c95bf865d9dfd5:

test restore/tpce/400GB/aws/nodes=9/cpus=8/zones=us-east-2b,us-west-2b,eu-west-1b was skipped due to (test_runner.go:725).runWorker: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2004-focal-v20210603 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --create-disk type=pd-ssd,size=1000GB,auto-delete=yes --machine-type n2-standard-8 --labels usage=roachtest,created=2023-07-13t18_06_39z,roachprod=true,cluster=teamcity-10892350-1689227049-150-n9cpu8-geo,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script2708143567 --project cockroach-ephemeral --boot-disk-size=32GB --zone eu-west-1b teamcity-10892350-1689227049-150-n9cpu8-geo-0007 teamcity-10892350-1689227049-150-n9cpu8-geo-0008 teamcity-10892350-1689227049-150-n9cpu8-geo-0009]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Permission denied on 'locations/eu-west-1b' (or it may not exist).: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=8 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ ac6d73de8e27cae4e087bb42ee03bfc515939104:

test kv0/enc=false/nodes=3/cpu=32/mt-shared-process was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-o4_izwlYVObqiwSGUpIkUs_-Dps= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-14-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T05:59:39Z},{Key=Name,Value=teamcity-10906784-1689313546-14-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-14-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T05:59:39Z},{Key=Name,Value=teamcity-10906784-1689313546-14-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script866655468 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping369391776 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ ac6d73de8e27cae4e087bb42ee03bfc515939104:

test kv0/enc=false/nodes=3/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-o4_izwlYVObqiwSGUpIkUs_-Dps= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T06:02:16Z},{Key=Name,Value=teamcity-10906784-1689313546-18-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-18-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T06:02:16Z},{Key=Name,Value=teamcity-10906784-1689313546-18-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-18-n4cpu32}] --user-data file:///tmp/aws-startup-script2138857448 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1464506967 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ ac6d73de8e27cae4e087bb42ee03bfc515939104:

test kv0/enc=false/nodes=1/cpu=32 was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-o4_izwlYVObqiwSGUpIkUs_-Dps= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T06:03:31Z},{Key=Name,Value=teamcity-10906784-1689313546-21-n2cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-21-n2cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T06:03:31Z},{Key=Name,Value=teamcity-10906784-1689313546-21-n2cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-21-n2cpu32}] --user-data file:///tmp/aws-startup-script361606067 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping181111176 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cluster_creation failed with artifacts on master @ ac6d73de8e27cae4e087bb42ee03bfc515939104:

test kv0/enc=false/nodes=3/cpu=32/size=64kb was skipped due to (test_runner.go:725).runWorker: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0a14db46282743a66 --key-name teamcity-runner-o4_izwlYVObqiwSGUpIkUs_-Dps= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-09065271eb9d0144d --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10906784-1689313546-25-n4cpu32-0003},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-25-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T06:06:03Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-10906784-1689313546-25-n4cpu32-0003},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-10906784-1689313546-25-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2023-07-14T06:06:03Z}] --user-data file:///tmp/aws-startup-script2154072139 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3850169418 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters: ROACHTEST_cloud=aws , ROACHTEST_cpu=32 , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #101289 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.1] - #87695 roachtest: cluster_creation failed [C-test-failure O-roachtest O-robot T-testeng branch-release-22.2]

This test on roachdash | Improve this report!