Open cockroach-teamcity opened 2 months ago
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test kv50/enc=false/nodes=4/cpu=96/batch=64 failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-15875273-1719726641-93-n5cpu96,lifetime=12h0m0s,arch=amd64,created=2024-06-30t13_25_49z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2d-custom-96-196608 --min-cpu-platform AMD Milan --metadata-from-file startup-script=/tmp/gce-startup-script2778136664 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-b teamcity-15875273-1719726641-93-n5cpu96-0001 teamcity-15875273-1719726641-93-n5cpu96-0002 teamcity-15875273-1719726641-93-n5cpu96-0003 teamcity-15875273-1719726641-93-n5cpu96-0004 teamcity-15875273-1719726641-93-n5cpu96-0005]
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-15875273-1719726641-93-n5cpu96-0001].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-15875273-1719726641-93-n5cpu96-0002].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-15875273-1719726641-93-n5cpu96-0003].
WARNING: Some requests generated warnings:
- Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
- The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240628'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded. Limit: 80000.0 in region us-east1.
metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
limit = 80000.0
dimensions = region: us-east1
vm_family: N2D
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=96
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-custom-32-65536
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-custom-32-65536 VM instance with 4 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:4
vmType: n2-custom-32-65536
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-custom-32-65536 VM instance with 4 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:4
vmType: n2-custom-32-65536
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=32
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test decommissionBench/nodes=6/warehouses=1000/drain-first/while-upreplicating/target=2/multi-region failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,created=2024-07-03t09_49_34z,roachprod=true,cluster=teamcity-15917012-1719986040-54-n8cpu16-geo,lifetime=12h0m0s,arch=amd64 --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script2878691603 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-b teamcity-15917012-1719986040-54-n8cpu16-geo-0005 teamcity-15917012-1719986040-54-n8cpu16-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-16
zone: us-central1-f
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-f' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-f zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-f
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-f' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-f zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-f
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-f' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test tpccbench/nodes=6/cpu=16/multi-az failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-15917012-1719986040-99-n7cpu16-geo,lifetime=12h0m0s,arch=amd64,created=2024-07-03t12_21_09z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script410047700 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-c teamcity-15917012-1719986040-99-n7cpu16-geo-0005 teamcity-15917012-1719986040-99-n7cpu16-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-c' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-c' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-d/instances/teamcity-15932324-1720072267-52-n7cpu2-0006].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-d/instances/teamcity-15932324-1720072267-52-n7cpu2-0007].
WARNING: Some requests generated warnings:
- Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
- The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240701'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-2 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-2
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-2 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-2
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=2
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test ledger/nodes=6/multi-az failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,arch=amd64,created=2024-07-04t10_53_09z,roachprod=true,cluster=teamcity-15932324-1720072267-89-n7cpu16-geo,lifetime=12h0m0s --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1337420786 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-b teamcity-15932324-1720072267-89-n7cpu16-geo-0003 teamcity-15932324-1720072267-89-n7cpu16-geo-0004]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test decommissionBench/nodes=6/warehouses=1000/drain-first/while-upreplicating/target=3/multi-region failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-15943268-1720158688-62-n8cpu16-geo,lifetime=12h0m0s,arch=amd64,created=2024-07-05t10_04_26z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script950660045 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-b teamcity-15943268-1720158688-62-n8cpu16-geo-0005 teamcity-15943268-1720158688-62-n8cpu16-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test ledger/nodes=6/multi-az failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,cluster=teamcity-15943268-1720158688-100-n7cpu16-geo,lifetime=12h0m0s,arch=amd64,created=2024-07-05t11_44_16z,roachprod=true --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script3673458806 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-b teamcity-15943268-1720158688-100-n7cpu16-geo-0003 teamcity-15943268-1720158688-100-n7cpu16-geo-0004]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-c, us-central1-f zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-c,us-central1-f
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-c, us-central1-f zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-c,us-central1-f
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-c, us-central1-f zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-c,us-central1-f
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-c, us-central1-f zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-c,us-central1-f
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=1
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
test ledger/nodes=6/multi-az failed: (test_runner.go:758).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-16005164-1720590611-87-n7cpu16-geo,lifetime=12h0m0s,arch=amd64,created=2024-07-10t11_56_36z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script2143867806 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-b teamcity-16005164-1720590611-87-n7cpu16-geo-0003 teamcity-16005164-1720590611-87-n7cpu16-geo-0004]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-23.2.7-rc @ 8a4e55e40110d752c158414bfce22846faeb1722:
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=96
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
Help
See: roachtest README
See: How To Investigate (internal)
Same failure on other branches
- #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125838 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
/cc @cockroachdb/test-eng
This test on roachdash | Improve this report!
Jira issue: CRDB-39837