Support Slurm (or other cluster management and job scheduling system)

mars-project / mars

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.

https://mars-project.readthedocs.io

Apache License 2.0

2.7k stars 325 forks source link

Open PeikaiLi opened 2 years ago

PeikaiLi commented 2 years ago

There are lots of school and company which use slurm or other cluster management and job scheduling system. It will be great if mars can support it.

qinxuye commented 2 years ago

This is sort of duplicate with #97 , we will discuss to see if we can support those scheduling systems.