TheBrownLab / PhyloFisher

PhyloFisher is a software package written in Python3 that can be used for the creation, analysis, and visualization of phylogenomic datasets that consist of eukaryotic protein sequences.
MIT License
31 stars 15 forks source link

RAxML within sgt_constructor uses all available resources #89

Closed anoukwillemsen closed 1 year ago

anoukwillemsen commented 1 year ago

Hello,

I am running sgt_constructor.py on a slurm compute cluster. I am limiting my run to 50 threads (see script below I use for the run), however, when RAxML is executed it starts using all available resources (In my case >1500 CPU cores, see attached image) on the specific node where its running. Of course this is not fair to other users on the cluster and makes sgt_constructor not suitable for running on a compute cluster. Could you please modify the sg_contructor.smk script (line 203) so that raxml-ng is limited to the resources specified?

Thanks and all the best,

Anouk

SLURM_workload_PhyloFisher

!/bin/bash

#

SBATCH --ntasks=5

SBATCH --ntasks-per-node=5

SBATCH --nodes=1

SBATCH --mem=150G

SBATCH --nice=1

SBATCH --partition=basic

SBATCH --time=3-20:00:00

SBATCH --output=log_phylofisher_GeneTree_4.out

SBATCH --error=log_phylofisher_GeneTree_4.err

export LC_ALL="C"; export LC_ALL="en_US.utf8";

load modules

module load conda; conda activate phylofisher-1.2.10;

module load ncbiblastplus/2.13.0; #################################

Gene Tree Construction

sgt_constructor.py -t 50 -i working_dataset_constructor_out_Jul.27.2023/ -o sgt_constructor_out_Aug.08.2023;

robert-ervin-jones commented 1 year ago

Hi Anouk,

I apologize for this. I did not realize raxml-ng used 8 threads by default. I will update the code so that raxml-ng only uses 1 thread.

Best, Robert

anoukwillemsen commented 1 year ago

Thanks for the update!