cooperative-computing-lab / cctools

The Cooperative Computing Tools (cctools) enable large scale distributed computations to harness hundreds to thousands of machines from clusters, clouds, and grids.
http://ccl.cse.nd.edu
Other
134 stars 120 forks source link

Create batch_queue_flux stub #3978

Open dblitt opened 1 week ago

dblitt commented 1 week ago

Proposed Changes

This PR creates a stub batch_queue module for the Flux Framework. I will be working in the immediate future to implement the submit, wait, and remove functionality of this module so Makeflow and vine_factory will be able to natively submit jobs to a flux instance. batch_queue_flux is currently marked as experimental.

I also have fixed some out of date documentation in batch_queue.h

Merge Checklist

The following items must be completed before PRs can be merge. Check these off to verify you have completed all steps.

tphung3 commented 1 week ago

Ok, this is a good template for further work. I think you know Flux enough by now to implement batch_queue_flux_submit and batch_queue_flux_remove. Don't worry about supporting all features, and instead try to submit basic Linux commands to flux and remove them using above APIs. To test your work you can write a custom test script calling batch_queue_flux_submit and batch_queue_flux_remove.