Closed nawrockie closed 5 months ago
Thanks so much for this Eric!
I think I know what the job_dequeuer refers to but will have to check. I will try to get back to you this week about the pull request, and if there are any issues. Thanks!
I think the job_dequeuer refers to this module https://github.com/Rfam/rfam-family-pipeline/blob/master/Rfam/Lib/Bio/Rfam/View/Dequeuer.pm, which if I understand is what runs the view process to update the website. Seems that it creates some LSF jobs, that could be replaced with SLURM commands. Could you look at modifying this? We have started a release this week so it will be a week or so until we can run the new slurm based code. Thanks!
@blakesweeney : yes I'll look into the dequeuer and get back to you if I have questions.
@blakesweeney and @emmaco : I took a look at the dequeuer code and it was written by John Tate about 9 years ago. It uses a LSF perl module that I'm not familiar with. I could try to morph it into working with slurm but it occurred to me that it may make sense to check with Pfam to see if they have already adapted a similar dequeuer from LSF to slurm, that is if they are still using a perl-based pipeline. Any idea?
I'll ask around and let you know.
@blakesweeney and @emmaco : MPI seems to be working now on slurm thanks to Ahmed Elmazaty's help. But for it to work with the pipeline, infernal will have to be re-configured with the --enable-mpi
flag after loading an openmpi module. I'm not sure exactly how you build infernal , but the following commands will build MPI-ready executables when run rom the top-level infernal src directory. (Also, you may want to upgrade to infernal 1.1.5.)
$ module load openmpi/4.1.4
$ which mpicc
/hps/software/spack/opt/spack/linux-rocky8-cascadelake/gcc-11.2.0/openmpi-4.1.4-jzdz5rzapd7n4z5z4rmdqm3dyyzxaglp/bin/mpicc
$ sh ./configure --enable-mpi; make
If you're wondering if/when MPI will be useful: my guess is it will help most for building and searching the biggest families (LSU, SSU) and also for generating alignments (rfmake.pl -a) for any big families with many thousands of hits in the FULL.
Hi Eric, we just finished up the release last week so we can try this out. Do you think upgrading infernal is required, or can we test out SLURM changes without updating infernal for now?
Non-MPI slurm functionality can be tested with existing infernal 1.1.4 as it is in /hps/software/users/agb/rfam/bin/
, if infernal 1.1.4 is recompiled following the instructions above with --enable-mpi
then you can test MPI functionality too. You could update to 1.1.5 at that point, but it's not necessary for testing at all.
Adds
scheduler
field toRfam/Conf/rfam.conf
which can either belsf
orslurm
, and adds support for it in the code, mainly inRfam/Lib/Bio/Rfam/Utils.pm
insubmit_nonmpi_job
,submit_mpi_job
, andwait_for_cluster_light
.I've tested anecdotally and it works in my hands with
scheduler
defined aslsf
(on LSF) orslurm
(on SLURM), with the exception of MPI.There may be some new minor edits we have to make for MPI compatibility. I'm getting help with systems currently on that. But MPI is not default on any scripts, and is only used if
-cmpi
(rfsearch) or-mpi
(rfmake) are enabled.There's two places I'm still unsure of:
In
Rfam/Scripts/view/rfam_family_view.pl
, the help message includes:I am unfamiliar with the job dequeuer so I'm not sure if anything
needs to be done to switch to slurm. Do either of you know more?
And in
Rfam/Scripts/view/make_sunburst.pl
the end of the help has anexample
bsub
command that explains how to process the families inchunks. I'm not familiar with this either so didn't try to update it
to an
sbatch
example, even though it probably should be.