search

ganga-devs / ganga

Ganga is an easy-to-use frontend for job definition and management
GNU General Public License v3.0
100 stars 159 forks source link

Support for SLURM backend in GangaAtlas #615

Closed will-cern closed 6 years ago

will-cern commented 8 years ago

Hello,

What would need to happen to get SLURM (https://rc.fas.harvard.edu/resources/running-jobs/) supported as a backend for athena jobs?

Cheers Will

drmarkwslater commented 8 years ago

Not a lot actually 😄 As an initial test, I believe you can just override the settings in the [PBS] section of .gangarc with the appropriate values for SLURM. I'm afraid it's difficult for us to test as we don't have an installation of this but if you find settings that work I can easily use these for another 'SLURM' backend.

drmarkwslater commented 8 years ago

@will-cern Did you make any progress with this? If you send me a typical transcript of job submission and monitoring in SLURM that would be enough I think...

will-cern commented 8 years ago

I wasn't actually the one who had access to a SLURM backend either, it was Alex Tuna, who I've asked if he will try out your suggestion. I am assuming that after putting the right values in the PBS section of .gangarc, he can submit a job in the same way as https://twiki.cern.ch/twiki/bin/view/AtlasProtected/AthAnalysisBase#How_to_submit_your_job_to_condor but replacing "condor" with "pbs"

drmarkwslater commented 8 years ago

@will-cern Exactly - the PBS backend is pretty generic so if the commands are setup correctly, then it should work out of the box. Let me know if he needs any help!

rob-c commented 8 years ago

@drmarkwslater There appears to be some interest from different people to using Ganga with SLURM, is it worth elevating this to a default backend configuration which we 'support' out of the box?

drmarkwslater commented 8 years ago

@rob-c That's the plan. The problem is finding a SLURM instance to test it against. If I have a day spare at some point I'll probably set one up myself and use that to get the backend code written.

schmitts commented 7 years ago

@drmarkwslater Any update on the slurm support? If not, I would start to implement and test it now.

unode commented 7 years ago

Would this SLURM image help for testing things out? Together with the wercker platform (free for open-source projects) I now frequently use docker images as part of the testing framework of some of my projects. There's also images for SGE, Torque and possibly others.

egede commented 7 years ago

@unode in principle an interesting option. I can't see how our development team (total size about 4 times 0.1 effort) will be able to do this any time soon. If somebody else would be able to integrate it into the testing framework then we are happy to assist.

mesmith75 commented 6 years ago

Slurm is in since #1192 .