Running more tests in parallel for faster presubmit outcome

GoogleCloudDataproc / initialization-actions

Run in all nodes of your cluster before the cluster starts - lets you customize your cluster

https://cloud.google.com/dataproc/init-actions

Apache License 2.0

588 stars 512 forks source link

Running more tests in parallel for faster presubmit outcome #1256

Closed Deependra-Patel closed 1 week ago

Deependra-Patel commented 2 weeks ago

Now the tests finish < 2 hours for 2.1-deb11 which were earlier timing out in 4 hours, so atleast 2x faster.

kubectl run doesn't allow specifying CPU/Memory requirements in single line. The test cluster uses Autopilot so resources will be used only when tests are run.

Deependra-Patel commented 2 weeks ago

/gcbrun

Deependra-Patel commented 1 week ago

Checked that some integ tests are failing due to unrelated reason which is being investigated separately by @prince-cs. None of them are due to quota issues which could have been caused with this change. Merging.