[testing] CodeFlare SDK with MCAD

project-codeflare / multi-cluster-app-dispatcher

Holistic job manager on Kubernetes

Apache License 2.0

106 stars 62 forks source link

[testing] CodeFlare SDK with MCAD #541

Open asm582 opened 11 months ago

asm582 commented 11 months ago

We need to fire tens of AWs with CodeFlare SDK to test queuing. The idea is to increase cluster utilization by queuing jobs and dispatching the next 1 as the previous jobs are complete.

Fire 100 MNIST jobs using CodeFlare SDK
Assuming the cluster has limited resources ie only to run 1 MNIST job, 99 of the jobs should be queued.
All jobs should run to completion

CC @astefanutti @anishasthana @Maxusmusti @dimakis

asm582 commented 11 months ago

@astefanutti @MichaelClifford @Fiona-Waters @anishasthana any updates that we would like to share on the above issue?