search

jonahcullen / wags

GNU General Public License v3.0
15 stars 6 forks source link

HPC scheduler is LSF :( #33

Open benyoung93 opened 1 year ago

benyoung93 commented 1 year ago

Good morning :).

First of all this tool looks extremely impressive and will allow streamlining of our pipelines that currently have way to main different jobs.

Unfortunately, the current HPC I have access to uses LSF for scheduling (sighhhhh). As I am a wee bit new to snakemake, how hard would it be to generate a --profile for LSF? IS this something you would have the time to do ?

If it is simple and there is a way for me to do it please let me know, any linking of articles/protocols for it I can give it a go.

Ben

jonahcullen commented 1 year ago

Hey Ben, I'm happy to try and generate a LSF profile for you (and whoever else lands here!). The only reason I have not yet as I don't have a quick way of checking it's completeness. So if you wanted to be the proverbial guinea pig I can let you know when I have a draft profile up and ready?

benyoung93 commented 1 year ago

Yes that sounds wonderful !!! thank you so much for the quick response. I am pretty free with work schedule so whenever you get this up I can test pretty quickly :).

Ben

jonahcullen commented 1 year ago

Can you run grep '^LSF_UNIT_FOR_LIMITS' ${LSF_ENVDIR}/lsf.conf from your HPC?

benyoung93 commented 1 year ago

Hi hi

Yes of course, output is as follows

LSF_UNIT_FOR_LIMITS=MB

I did have a quick half an hour attempt at this https://github.com/Snakemake-Profiles/lsf which also asked for the command you pasted above. I successfully created a profile but could not get it to work with wags :(

jonahcullen commented 12 months ago

Hey @benyoung93 I did not forget about this! I'll get back to you yet this week for you to test it at your institution.

jonahcullen commented 10 months ago

Hey again @benyoung93 I unfortunately did forget as I was finishing my thesis and preparing for the final defense. This is back on my radar now. No worries if you moved on, I'll still put it together if you want to test it out later!