Open cockroach-teamcity opened 2 months ago
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 4037be1494ff8f3873edfa2510a7fa79545b413e:
vmType: n2-standard-4
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
test costfuzz/seed-multi-region failed: (test_runner.go:797).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-15853648-1719553745-42-n9cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2024-06-28t07_39_34z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script3887617497 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-d teamcity-15853648-1719553745-42-n9cpu4-geo-0007]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
vmType: n2-standard-16
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
vmType: n2-standard-16
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:797).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-15953524-1720245229-105-n4cpu96,lifetime=12h0m0s,arch=amd64,created=2024-07-06t11_57_11z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2d-custom-96-196608 --min-cpu-platform AMD Milan --metadata-from-file startup-script=/tmp/gce-startup-script92092193 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-b teamcity-15953524-1720245229-105-n4cpu96-0001 teamcity-15953524-1720245229-105-n4cpu96-0002 teamcity-15953524-1720245229-105-n4cpu96-0003 teamcity-15953524-1720245229-105-n4cpu96-0004]
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-15953524-1720245229-105-n4cpu96-0001].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-15953524-1720245229-105-n4cpu96-0002].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-b/instances/teamcity-15953524-1720245229-105-n4cpu96-0004].
WARNING: Some requests generated warnings:
- Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
- The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240701'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Quota 'N2D_CPUS' exceeded. Limit: 1000.0 in region us-east1.
metric name = compute.googleapis.com/n2d_cpus
limit name = N2D-CPUS-per-project-region
limit = 1000.0
dimensions = region: us-east1
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=96
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone, because of local_ssd_availability root cause(s).
Consider trying your request in the us-central1-a, us-central1-f, us-central1-c
zone(s), which currently has capacity to accommodate your request. Alternatively,
you can try your request again with a different VM hardware configuration or
at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: local_ssd_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone, because of local_ssd_availability root cause(s).
Consider trying your request in the us-central1-a, us-central1-f, us-central1-c
zone(s), which currently has capacity to accommodate your request. Alternatively,
you can try your request again with a different VM hardware configuration or
at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: local_ssd_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-c, us-central1-f zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-c,us-central1-f
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-c, us-central1-f zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-c,us-central1-f
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone, because of local_ssd_availability root cause(s).
Consider trying your request in the us-central1-a, us-central1-f, us-central1-c
zone(s), which currently has capacity to accommodate your request. Alternatively,
you can try your request again with a different VM hardware configuration or
at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: local_ssd_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone, because of local_ssd_availability root cause(s).
Consider trying your request in the us-central1-a, us-central1-f, us-central1-c
zone(s), which currently has capacity to accommodate your request. Alternatively,
you can try your request again with a different VM hardware configuration or
at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: local_ssd_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-central1-b zone. Consider trying your request in the us-central1-a,
us-central1-f, us-central1-c zone(s), which currently has capacity to accommodate
your request. Alternatively, you can try your request again with a different
VM hardware configuration or at a later time. For more information, see the
troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-central1-b
zonesAvailable: us-central1-a,us-central1-f,us-central1-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
test costfuzz/seed-multi-region failed: (test_runner.go:797).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,roachprod=true,cluster=teamcity-16005194-1720590572-43-n9cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2024-07-10t08_37_41z --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script2079153566 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-d teamcity-16005194-1720590572-43-n9cpu4-geo-0007]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
test decommissionBench/nodes=6/warehouses=1000/drain-first/while-upreplicating/target=3/multi-region failed: (test_runner.go:797).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,created=2024-07-10t09_01_27z,roachprod=true,cluster=teamcity-16005194-1720590572-51-n8cpu16-geo,lifetime=12h0m0s,arch=amd64 --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2-standard-16 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1585014803 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-central1-b teamcity-16005194-1720590572-51-n8cpu16-geo-0005 teamcity-16005194-1720590572-51-n8cpu16-geo-0006]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-16 VM instance with 2 local SSD(s) is currently unavailable
in the us-central1-b zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:2
vmType: n2-standard-16
zone: us-central1-b
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-central1-b' does not have
enough resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=16
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
- The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240701'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east4-b zone. Consider trying your request in the us-east4-a, us-east4-c
zone(s), which currently has capacity to accommodate your request. Alternatively,
you can try your request again with a different VM hardware configuration or
at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east4-b
zonesAvailable: us-east4-a,us-east4-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east4-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east4-b zone. Consider trying your request in the us-east4-a, us-east4-c
zone(s), which currently has capacity to accommodate your request. Alternatively,
you can try your request again with a different VM hardware configuration or
at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east4-b
zonesAvailable: us-east4-a,us-east4-c
reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east4-b' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
vmType: n2-standard-8
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-8 VM instance with 8 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:8
vmType: n2-standard-8
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-8 VM instance with 8 local SSD(s) is currently unavailable
in the us-east1-d zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:8
vmType: n2-standard-8
zone: us-east1-d
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=8
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=8
See: roachtest README
See: How To Investigate (internal)
- #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
Output: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-c/instances/teamcity-16040712-1720763424-104-n4cpu4-0001].
Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-east1-c/instances/teamcity-16040712-1720763424-104-n4cpu4-0004].
WARNING: Some requests generated warnings:
- Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
- The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230727' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240701'.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 7e81be6de75205c3d08b0d8dcc6ca188306abc27:
test unoptimized-query-oracle/disable-rules=half/seed-multi-region failed: (test_runner.go:797).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels usage=roachtest,arch=amd64,created=2024-07-12t11_48_16z,roachprod=true,cluster=teamcity-16040712-1720763424-125-n9cpu4-geo,lifetime=12h0m0s --scopes cloud-platform --image ubuntu-2204-jammy-v20230727 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --maintenance-policy MIGRATE --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script3386983943 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-c teamcity-16040712-1720763424-125-n9cpu4-geo-0001]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A n2-standard-4 VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: n2-standard-4
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
- #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #126641 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.24-rc] - #126293 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc] - #125986 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.23-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
roachtest.cluster_creation failed with artifacts on release-24.1.2-rc @ 4037be1494ff8f3873edfa2510a7fa79545b413e:
Parameters:
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=96
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
Help
See: roachtest README
See: How To Investigate (internal)
Same failure on other branches
- #125680 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.6-rc] - #123744 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #123453 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.21-rc] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #120814 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.4-rc] - #120620 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108859 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-22.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
/cc @cockroachdb/test-eng
This test on roachdash | Improve this report!
Jira issue: CRDB-39629