roachtest: cluster_creation failed

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv0/enc=false/nodes=3/cpu=32/mvcc-range-keys=global was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-13776800-1706769061-17-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T06:49:12Z},{Key=Name,Value=teamcity-13776800-1706769061-17-n4cpu32-0001},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-13776800-1706769061-17-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T06:49:12Z},{Key=Name,Value=teamcity-13776800-1706769061-17-n4cpu32-0001},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3221720729 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2073028201 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-35832

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv0/enc=false/nodes=3/cpu=32/seq was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-20-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T06:51:06Z},{Key=Name,Value=teamcity-13776800-1706769061-20-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-20-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T06:51:06Z},{Key=Name,Value=teamcity-13776800-1706769061-20-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script2449719850 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping591089063 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv0/enc=false/nodes=3/cpu=32/size=4kb was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-13776800-1706769061-23-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T06:53:01Z},{Key=Name,Value=teamcity-13776800-1706769061-23-n4cpu32-0001},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-13776800-1706769061-23-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T06:53:01Z},{Key=Name,Value=teamcity-13776800-1706769061-23-n4cpu32-0001},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script609972485 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2133240555 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv0/enc=false/nodes=3/cpu=32/size=64kb was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-02-01T06:58:35Z},{Key=Name,Value=teamcity-13776800-1706769061-29-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-29-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-02-01T06:58:35Z},{Key=Name,Value=teamcity-13776800-1706769061-29-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-29-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script597250619 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3806545166 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv95/enc=false/nodes=3/cpu=32/seq was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-02-01T07:41:48Z},{Key=Name,Value=teamcity-13776800-1706769061-36-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-36-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-02-01T07:41:48Z},{Key=Name,Value=teamcity-13776800-1706769061-36-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-36-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script2360905041 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2992192679 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv95/enc=false/nodes=3/cpu=32/size=4kb was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-13776800-1706769061-39-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-39-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T07:43:49Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-13776800-1706769061-39-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-39-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T07:43:49Z}] --user-data file:///tmp/aws-startup-script3513861405 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2471639717 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv95/enc=false/nodes=3/cpu=32/size=64kb was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-x4QYpKXgvUfL_7Q8onaN-qCKsw0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-13776800-1706769061-42-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-42-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T07:45:44Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-13776800-1706769061-42-n4cpu32-0001},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-13776800-1706769061-42-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-01T07:45:44Z}] --user-data file:///tmp/aws-startup-script916470789 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1740223032 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-d, us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-d,us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-d, us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-d,us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-d, us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-d,us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-d, us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-d,us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-d, us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-d,us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-d, us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-d,us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

test kv0/enc=false/nodes=3/cpu=96 was skipped due to (test_runner.go:729).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-highcpu-96 --labels usage=roachtest,created=2024-02-06t00_41_48z,roachprod=true,cluster=teamcity-13840986-1707114634-162-n4cpu96,lifetime=12h0m0s,arch=amd64 --metadata-from-file startup-script=/tmp/gce-startup-script3255382195 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-b teamcity-13840986-1707114634-162-n4cpu96-0001 teamcity-13840986-1707114634-162-n4cpu96-0002 teamcity-13840986-1707114634-162-n4cpu96-0003 teamcity-13840986-1707114634-162-n4cpu96-0004]
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-13840986-1707114634-162-n4cpu96-0003].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-13840986-1707114634-162-n4cpu96-0004].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-13840986-1707114634-162-n4cpu96-0001].
WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240126'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 6b0574d5f66597760e0f0863df4aec8b4e65398c:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2-highcpu-96 VM instance with 16 local SSD(s) is currently unavailable
      in the us-east1-b zone. Consider trying your request in the us-east1-c, us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2-highcpu-96
      zone: us-east1-b
      zonesAvailable: us-east1-c,us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=96
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 558a499c40a72ad47b2b1791917da214e16f07d3:

test c2c/kv0 was skipped due to (test_runner.go:729).func4: cluster.PutE: put /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64 failed: ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64 ubuntu@35.237.116.162:./cockroach
ssh: connect to host 35.237.116.162 port 22: Connection timed out
lost connection: exit status 1

Parameters:

• ROACHTEST_arch=amd64
• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=8
• ROACHTEST_encrypted=false
• ROACHTEST_fs=ext4
• ROACHTEST_localSSD=false
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 558a499c40a72ad47b2b1791917da214e16f07d3:

test decommissionBench/nodes=6/warehouses=1000/drain-first/while-upreplicating/target=2/multi-region was skipped due to (test_runner.go:729).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --labels usage=roachtest,roachprod=true,cluster=teamcity-13860943-1707201015-100-n8cpu16-geo,lifetime=12h0m0s,arch=amd64,created=2024-02-06t18_08_58z --metadata-from-file startup-script=/tmp/gce-startup-script3887207718 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-13860943-1707201015-100-n8cpu16-geo-0003 teamcity-13860943-1707201015-100-n8cpu16-geo-0004 teamcity-13860943-1707201015-100-n8cpu16-geo-0008]
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-13860943-1707201015-100-n8cpu16-geo-0004].
WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240126'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
    metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
    limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
    limit = 30000.0
    dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=16
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 558a499c40a72ad47b2b1791917da214e16f07d3:

test decommissionBench/nodes=6/warehouses=1000/drain-first/while-upreplicating/target=3/multi-region was skipped due to (test_runner.go:729).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --labels usage=roachtest,roachprod=true,cluster=teamcity-13860943-1707201015-110-n8cpu16-geo,lifetime=12h0m0s,arch=amd64,created=2024-02-06t18_18_41z --metadata-from-file startup-script=/tmp/gce-startup-script3179554232 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-13860943-1707201015-110-n8cpu16-geo-0003 teamcity-13860943-1707201015-110-n8cpu16-geo-0004 teamcity-13860943-1707201015-110-n8cpu16-geo-0008]
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-13860943-1707201015-110-n8cpu16-geo-0003].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-13860943-1707201015-110-n8cpu16-geo-0008].
WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240126'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
    metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
    limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
    limit = 30000.0
    dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=16
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 558a499c40a72ad47b2b1791917da214e16f07d3:

test follower-reads/survival=region/locality=regional/reads=bounded-staleness was skipped due to (test_runner.go:729).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --labels usage=roachtest,arch=amd64,created=2024-02-06t18_19_38z,roachprod=true,cluster=teamcity-13860943-1707201015-111-n6cpu4-geo,lifetime=12h0m0s --metadata-from-file startup-script=/tmp/gce-startup-script1046035865 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-13860943-1707201015-111-n6cpu4-geo-0004 teamcity-13860943-1707201015-111-n6cpu4-geo-0005]
Output: WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240126'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
    metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
    limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
    limit = 30000.0
    dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 558a499c40a72ad47b2b1791917da214e16f07d3:

test tpcc/multiregion/survive=region/chaos=true was skipped due to (test_runner.go:729).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --labels usage=roachtest,lifetime=12h0m0s,arch=amd64,created=2024-02-06t21_50_58z,roachprod=true,cluster=teamcity-13860943-1707201015-164-n10cpu4-geo --metadata-from-file startup-script=/tmp/gce-startup-script2254037333 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-13860943-1707201015-164-n10cpu4-geo-0004 teamcity-13860943-1707201015-164-n10cpu4-geo-0005 teamcity-13860943-1707201015-164-n10cpu4-geo-0006]
Output: WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240126'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
    metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
    limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
    limit = 30000.0
    dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 558a499c40a72ad47b2b1791917da214e16f07d3:

test tpccbench/nodes=9/cpu=4/multi-region was skipped due to (test_runner.go:729).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-highmem-4 --labels usage=roachtest,roachprod=true,cluster=teamcity-13860943-1707201015-166-n12cpu4hm-geo,lifetime=12h0m0s,arch=amd64,created=2024-02-06t21_53_55z --metadata-from-file startup-script=/tmp/gce-startup-script2686899186 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-13860943-1707201015-166-n12cpu4hm-geo-0005 teamcity-13860943-1707201015-166-n12cpu4hm-geo-0006 teamcity-13860943-1707201015-166-n12cpu4hm-geo-0007 teamcity-13860943-1707201015-166-n12cpu4hm-geo-0008]
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-13860943-1707201015-166-n12cpu4hm-geo-0005].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-13860943-1707201015-166-n12cpu4hm-geo-0006].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-13860943-1707201015-166-n12cpu4hm-geo-0008].
WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240126'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
    metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
    limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
    limit = 30000.0
    dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #115827 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.0-rc] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test kv0/enc=false/nodes=3/cpu=32/seq was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-SuoBLX2ysfJ3lQAhEMFfKCLYauM= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-14023258-1708065040-18-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-16T06:49:22Z},{Key=Name,Value=teamcity-14023258-1708065040-18-n4cpu32-0001},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-14023258-1708065040-18-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-16T06:49:22Z},{Key=Name,Value=teamcity-14023258-1708065040-18-n4cpu32-0001},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3381331291 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2006736325 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test kv0/enc=false/nodes=3/cpu=32/size=4kb was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-SuoBLX2ysfJ3lQAhEMFfKCLYauM= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-14023258-1708065040-21-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-16T06:51:06Z},{Key=Name,Value=teamcity-14023258-1708065040-21-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-14023258-1708065040-21-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-16T06:51:06Z},{Key=Name,Value=teamcity-14023258-1708065040-21-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script3473995304 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2062682564 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test kv0/enc=false/nodes=3/cpu=32/size=64kb was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-SuoBLX2ysfJ3lQAhEMFfKCLYauM= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-02-16T06:56:26Z},{Key=Name,Value=teamcity-14023258-1708065040-27-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-14023258-1708065040-27-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-02-16T06:56:26Z},{Key=Name,Value=teamcity-14023258-1708065040-27-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-14023258-1708065040-27-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script1308357077 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping129437269 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test kv0/enc=false/nodes=3/cpu=32/splt=0 was skipped due to (test_runner.go:729).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-SuoBLX2ysfJ3lQAhEMFfKCLYauM= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0e71591c1fe06645e --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-14023258-1708065040-30-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-14023258-1708065040-30-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-16T06:58:13Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-14023258-1708065040-30-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-14023258-1708065040-30-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-02-16T06:58:13Z}] --user-data file:///tmp/aws-startup-script3183799163 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping730865892 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2c). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2b.: exit status 254

Parameters:

• ROACHTEST_cloud=aws
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=32
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test failover/non-system/disk-stall/lease=expiration was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=2
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test failover/partial/lease-gateway was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=2
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test failover/partial/lease-liveness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=2
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test failover/partial/lease-gateway/lease=expiration was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=2
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test failover/partial/lease-liveness/lease=expiration was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=2
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/mixed-version/survival=region/locality=global/reads=strong was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=region/locality=global/reads=bounded-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=region/locality=global/reads=exact-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=region/locality=global/reads=strong was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=region/locality=regional/reads=bounded-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=region/locality=regional/reads=bounded-staleness/insufficient-quorum was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=region/locality=regional/reads=exact-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=zone/locality=global/reads=bounded-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=zone/locality=global/reads=exact-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=zone/locality=global/reads=strong was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=zone/locality=regional/reads=bounded-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=zone/locality=regional/reads=bounded-staleness/insufficient-quorum was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test follower-reads/survival=zone/locality=regional/reads=exact-staleness was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test import/tpch/nodes=8 was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test jepsen/bank-multitable/majority-ring was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test jepsen/bank-multitable/majority-ring-start-kill-2 was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test jepsen/bank-multitable/majority-ring-subcritical-skews was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test clearrange/checks=false was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=16
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-23.2.1-rc @ 898cd6a363fd47bb92a03bac216f9bed0f64bc08:

test jepsen/bank-multitable/parts-start-kill-2 was skipped due to (test_runner.go:729).func4: failed to retrieve authorized keys from gcloud: malformed public key line "---"

Parameters:

• ROACHTEST_cloud=gce
• ROACHTEST_coverageBuild=false
• ROACHTEST_cpu=4
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0
  Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #118663 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.15-rc] - #115949 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

[renatolabs]

Branch deleted, closing.