Open cockroach-teamcity opened 6 months ago
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test perturbation/metamorphic/addNode failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on azure [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=8
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test perturbation/metamorphic/decommission failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on azure [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=2
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test perturbation/metamorphic/restart failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on azure [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=8
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test schemachange/random-load failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test unoptimized-query-oracle/disable-rules=all/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test unoptimized-query-oracle/disable-rules=all/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar issue without the B-runtime-assertions-enabled label. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test unoptimized-query-oracle/disable-rules=half/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test unoptimized-query-oracle/disable-rules=half/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test follower-reads/mixed-version/survival=region/locality=global/reads=strong failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,created=2024-09-03t12_10_45z,roachprod=true,cluster=teamcity-16726095-1725342640-126-n6cpu4-geo,lifetime=12h0m0s,arch=amd64 --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1837607275 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16726095-1725342640-126-n6cpu4-geo-0004 teamcity-16726095-1725342640-126-n6cpu4-geo-0005]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test unoptimized-query-oracle/disable-rules=all/seed-multi-region failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,lifetime=12h0m0s,arch=amd64,created=2024-09-03t13_07_18z,roachprod=true,cluster=teamcity-16726095-1725342640-171-n9cpu4-geo --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1271713062 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16726095-1725342640-171-n9cpu4-geo-0002]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test unoptimized-query-oracle/disable-rules=half/rand-multi-region failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-16726095-1725342640-172-n9cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2024-09-03t13_07_25z --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1627681791 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16726095-1725342640-172-n9cpu4-geo-0002]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:
test perturbation/metamorphic/partition failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on azure [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=32
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test perturbation/metamorphic/addNode failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on aws [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test perturbation/metamorphic/restart failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on aws [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=32
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test restore/tpce/400GB/aws/nodes=4/cpus=16 failed: (test_runner.go:841).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-Te_i1KEYg0S0wz2MBL8F6s42dE0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16743678-1725428970-57-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2024-09-04T07:22:10Z},{Key=Name,Value=teamcity-16743678-1725428970-57-n4cpu16-0002},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16743678-1725428970-57-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2024-09-04T07:22:10Z},{Key=Name,Value=teamcity-16743678-1725428970-57-n4cpu16-0002},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script2964038653 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3288615691 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr:
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m7g.4xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m7g.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test ycsb/B/nodes=3/cpu=32 failed: (test_runner.go:841).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-Te_i1KEYg0S0wz2MBL8F6s42dE0= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-16743678-1725428970-66-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16743678-1725428970-66-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-09-04T07:36:50Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-16743678-1725428970-66-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16743678-1725428970-66-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-09-04T07:36:50Z}] --user-data file:///tmp/aws-startup-script2436720321 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1351291087 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr:
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=32
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test costfuzz/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar issue without the B-runtime-assertions-enabled label. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test costfuzz/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test schemachange/random-load failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test unoptimized-query-oracle/disable-rules=all/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar issue without the B-runtime-assertions-enabled label. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test unoptimized-query-oracle/disable-rules=all/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test unoptimized-query-oracle/disable-rules=half/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:
test unoptimized-query-oracle/disable-rules=half/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test kv95/enc=false/nodes=3/tracing failed: (test_runner.go:841).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-KYR8IkQxSZqfOYOTsCvgr2quK7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2024-09-05T07:08:21Z},{Key=Name,Value=teamcity-16760065-1725515315-53-n4cpu8-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16760065-1725515315-53-n4cpu8},{Key=Lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2024-09-05T07:08:21Z},{Key=Name,Value=teamcity-16760065-1725515315-53-n4cpu8-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16760065-1725515315-53-n4cpu8},{Key=Lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script1591896929 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1233019796 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr:
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=8
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test perturbation/metamorphic/partition failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on aws [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=8
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test perturbation/metamorphic/addNode failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on aws [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test costfuzz/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test costfuzz/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test costfuzz/rand-multi-region failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,created=2024-09-05t09_52_23z,roachprod=true,cluster=teamcity-16760064-1725515361-86-n9cpu4-geo,lifetime=12h0m0s,arch=amd64 --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script837311276 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16760064-1725515361-86-n9cpu4-geo-0002]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test costfuzz/seed-multi-region failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-16760064-1725515361-89-n9cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2024-09-05t09_59_34z --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script484819201 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16760064-1725515361-89-n9cpu4-geo-0002]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test schemachange/random-load failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test indexes/2/nodes=6/multi-region failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,arch=amd64,created=2024-09-05t11_45_27z,roachprod=true,cluster=teamcity-16760064-1725515361-124-n7cpu16-geo,lifetime=12h0m0s --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script4146608126 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16760064-1725515361-124-n7cpu16-geo-0003 teamcity-16760064-1725515361-124-n7cpu16-geo-0004]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test unoptimized-query-oracle/disable-rules=all/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar issue without the B-runtime-assertions-enabled label. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test unoptimized-query-oracle/disable-rules=all/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test unoptimized-query-oracle/disable-rules=half/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test unoptimized-query-oracle/disable-rules=half/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test perturbation/metamorphic/restart failed: (test_runner.go:841).func4: specifying SSD count is not yet supported on azure [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=32
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ b6862f42d3a229cad814b61d5a77a420afb540b5:
test unoptimized-query-oracle/disable-rules=all/seed-multi-region failed: (test_runner.go:841).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,arch=amd64,created=2024-09-05t13_23_26z,roachprod=true,cluster=teamcity-16760064-1725515361-183-n9cpu4-geo,lifetime=12h0m0s --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script4261972675 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-16760064-1725515361-183-n9cpu4-geo-0002]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-west1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-west1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-west1-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #129712 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc] - #129301 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #128956 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.10-rc] - #128942 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.4-rc] - #128531 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.25-rc] - #127332 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-24.1.3-rc] - #127326 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.9-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:841).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-Nv8QTXusVCCXsGpPN8Ap8Sw0ovU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-09-06T07:45:31Z},{Key=Name,Value=teamcity-16774771-1725601768-45-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16774771-1725601768-45-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2024-09-06T07:45:31Z},{Key=Name,Value=teamcity-16774771-1725601768-45-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-16774771-1725601768-45-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script1831555794 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping732900263 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr:
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6a.24xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6a.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]
Parameters:
ROACHTEST_cloud=aws
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=96
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test costfuzz/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test costfuzz/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test costfuzz/rand-multi-region failed: (test_runner.go:841).func4: in provider: gce: T2A instances are not supported outside of [asia-southeast1-b,asia-southeast1-c,europe-west4-a,europe-west4-b,europe-west4-c,us-central1-a,us-central1-b,us-central1-f] [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test schemachange/random-load failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test unoptimized-query-oracle/disable-rules=all/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test unoptimized-query-oracle/disable-rules=all/seed-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:
test unoptimized-query-oracle/disable-rules=half/rand-multi-region failed: (test_runner.go:841).func4: one or more parallel execution failure(s): COMMAND_PROBLEM: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #130222 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130221 roachtest: cluster_creation failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-23.2] - #130220 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #130219 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130218 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130217 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130216 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130215 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130214 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #130213 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on master @ e90d15c9807775a1a1d89f75cac4c9383c325709:
Parameters:
ROACHTEST_cloud=azure
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=32
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
Help
See: roachtest README
See: How To Investigate (internal)
/cc @cockroachdb/test-engThis test on roachdash | Improve this report!
Jira issue: CRDB-38523