search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.99k stars 3.79k forks source link

roachtest: cluster_creation failed #131888

Open cockroach-teamcity opened 5 days ago

cockroach-teamcity commented 5 days ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv0/enc=false/nodes=3/cpu=32 failed: (test_runner.go:821).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-trpPqh6FOv8HihB2VKAw4r4iv44= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-17162325-1728020899-20-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-10-04T06:12:00Z},{Key=Name,Value=teamcity-17162325-1728020899-20-n4cpu32-0003},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-17162325-1728020899-20-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-10-04T06:12:00Z},{Key=Name,Value=teamcity-17162325-1728020899-20-n4cpu32-0003},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script2197376261 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2219771187 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-42737

cockroach-teamcity commented 2 days ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test lib/pq failed: (test_runner.go:821).func4: in provider: azure: creating VM teamcity-17202433-1728280076-318-n1cpu4-0001: creating IP teamcity-17202433-1728280076-318-n1cpu4-0001: network.PublicIPAddressesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPCountLimitReached" Message="Cannot create more than 1000 public IP addresses for this subscription in this region." Details=[] [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 days ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test ldr/kv0/workload=both/ingestion=both failed: (test_runner.go:821).func4: in provider: azure: for location "eastus": creating Azure VNet "common" in "roachprod-vnets-eastus": network.VirtualNetworksClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="RetryableError" Message="A retryable error occurred." Details=[{"code":"RetryableErrorDueToAnotherOperation","message":"Operation PutVirtualNetworkOperation (895b3aa3-109f-46c0-a57c-bc5b08bcaf98) is updating resource /subscriptions/73075e4f-150d-4f4f-956b-053507b61fa7/resourceGroups/roachprod-vnets-eastus/providers/Microsoft.Network/virtualNetworks/roachprod-vnets-eastus. The call can be retried in 14 seconds."}] [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 days ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test loqrecovery/half-online/workload=tpcc/rangeSize=16mb failed: (test_runner.go:821).func4: in provider: azure: creating VM teamcity-17202433-1728280076-325-n6cpu4-0002: creating IP teamcity-17202433-1728280076-325-n6cpu4-0002: network.PublicIPAddressesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPCountLimitReached" Message="Cannot create more than 1000 public IP addresses for this subscription in this region." Details=[] [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 days ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test multitenant/shared-process/basic failed: (test_runner.go:821).func4: in provider: azure: creating VM teamcity-17202433-1728280076-329-n5cpu4-0003: creating IP teamcity-17202433-1728280076-329-n5cpu4-0003: network.PublicIPAddressesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPCountLimitReached" Message="Cannot create more than 1000 public IP addresses for this subscription in this region." Details=[] [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 days ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test loqrecovery/half-online/workload=movr/rangeSize=2mb failed: (test_runner.go:821).func4: in provider: azure: creating VM teamcity-17202433-1728280076-322-n6cpu4-0002: creating IP teamcity-17202433-1728280076-322-n6cpu4-0002: network.PublicIPAddressesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPCountLimitReached" Message="Cannot create more than 1000 public IP addresses for this subscription in this region." Details=[] [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv0/enc=false/nodes=3/cpu=96 failed: (test_runner.go:821).func4: in provider: aws: machine type m7gd.xlarge is arm64, but requested arch is amd64 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv95/enc=false/nodes=3/cpu=32 failed: (test_runner.go:821).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-ASU-ddvWj-k-_ZUIBSmweAfz4KY= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-10-08T06:15:34Z},{Key=Name,Value=teamcity-17218403-1728366405-27-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-17218403-1728366405-27-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-10-08T06:15:34Z},{Key=Name,Value=teamcity-17218403-1728366405-27-n4cpu32-0002},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-17218403-1728366405-27-n4cpu32}] --user-data file:///tmp/aws-startup-script3891580583 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1814526095 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:821).func4: in provider: aws: machine type m7gd.xlarge is arm64, but requested arch is amd64 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test ycsb/C/nodes=3/cpu=32 failed: (test_runner.go:821).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-ASU-ddvWj-k-_ZUIBSmweAfz4KY= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-17218403-1728366405-54-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-10-08T06:54:37Z},{Key=Name,Value=teamcity-17218403-1728366405-54-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-17218403-1728366405-54-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2024-10-08T06:54:37Z},{Key=Name,Value=teamcity-17218403-1728366405-54-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script472613390 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3738750907 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-c zone. Consider trying your request in the us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-c
      zonesAvailable: us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-c zone. Consider trying your request in the us-east1-d
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-c
      zonesAvailable: us-east1-d
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:821).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --labels spot=true,usage=roachtest,roachprod=true,cluster=teamcity-17218402-1728366392-109-n4cpu96,lifetime=12h0m0s,arch=amd64,created=2024-10-08t11_49_35z --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account 21965078311-compute@developer.gserviceaccount.com --provisioning-model SPOT --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --local-ssd interface=NVME --machine-type n2d-custom-96-196608 --min-cpu-platform AMD Milan --metadata-from-file startup-script=/tmp/gce-startup-script1131412195 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-east1-d teamcity-17218402-1728366392-109-n4cpu96-0001 teamcity-17218402-1728366392-109-n4cpu96-0002 teamcity-17218402-1728366392-109-n4cpu96-0003]
Output: ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 80000.0 in region us-east1.
    metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
    limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
    limit = 80000.0
    dimensions = region: us-east1
vm_family: N2D
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 16 hours ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:821).func4: in provider: aws: machine type m7gd.xlarge is arm64, but requested arch is amd64 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 12 hours ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test kv0/enc=false/nodes=3/cpu=32 failed: (test_runner.go:821).func4: in provider: gce: machine type t2a-standard-4 is arm64, but requested arch is amd64 [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!

cockroach-teamcity commented 11 hours ago

roachtest.cluster_creation failed with artifacts on release-24.2.4-rc @ 8140b8e294fb16a293d0aae662bcf5c1d488c852:

test sqlsmith/setup=seed-multi-region/setting=multi-region failed: (test_runner.go:821).func4: in provider: gce: T2A instances are not supported outside of [asia-southeast1-b,asia-southeast1-c,europe-west4-a,europe-west4-b,europe-west4-c,us-central1-a,us-central1-b,us-central1-f] [owner=test-eng]

Parameters:

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

- #131730 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.6-rc] - #131729 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.28-rc] - #131725 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.13-rc] - #126981 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #123745 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master] - #121332 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #112176 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2] - #108533 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]

This test on roachdash | Improve this report!