search

sampsyo / cluster-workers

a Python client/master/worker system for distributing jobs in a cluster
44 stars 18 forks source link

Condor support #1

Open ekfriis opened 11 years ago

ekfriis commented 11 years ago

Hello,

I am on a quest to find a painless way to submit jobs to our condor cluster. I found clusterfutures (it is very nice, thanks!), and have gotten it running on our cluster at UW.

After the fact, I found cluster-workers. I appreciate the philosophy of not spinning up a job for each task. Do you see any big problems with implementing the equivalent of slurm.py for condor? If not, perhaps I will take a crack at it.

Evan

sampsyo commented 11 years ago

This is a great idea. Our cluster originally ran Condor but we switched over to Slurm last year in an OS change. I see no reason why Condor shouldn't work equally well with the cluster-workers approach. Let me know if you find any lingering Slurm-specific code -- I'll try to help make everything as generic as possible.

Greetings from the other UW,

Adrian