search

amkozlov / raxml-ng

RAxML Next Generation: faster, easier-to-use and more flexible
GNU Affero General Public License v3.0
383 stars 64 forks source link

alternative parallelization (multiple concurrent tree searches/bootstraps, like in RAxML-MPI) #8

Closed amkozlov closed 4 years ago

MaximilianStammnitz commented 4 years ago

Hi @amkozlov, wanted to ask if there's already a raxml-ng test version for this intended implementation of concurrent tree searches/bootstraps? I'd be keen to give it a try if so.

Many thanks, Max

amkozlov commented 4 years ago

Hi @MaximilianStammnitz, yes there is a test version available here:

https://github.com/amkozlov/raxml-ng/tree/coarse

You can add --workers N option where N is the desired number of concurrent tree searches. Please note that not everything works in this version yet, eg. MPI+checkpointing (and MPI support in general is not very well tested). I plan to finalize coarse-grain parallelization support in the next few weeks, but of course early adopters/testers are always welcome! :)

MaximilianStammnitz commented 4 years ago

Great, many thanks! Happy to try it out and we don't use MPI, so that shouldn't be a big deal for now.

Could you possibly also make the linux_x86_64 binary available for this fork? (sorry for my lack of experience with this, hope this would not take too much time).

Cheers, Max

amkozlov commented 4 years ago

sure, here you go:

raxng-coarse.zip

MaximilianStammnitz commented 4 years ago

Thanks, running it now and will report back.

Max

amkozlov commented 4 years ago

implemented in the coarse branch, to be included in raxml-ng 1.0

cgjosephlee commented 4 years ago

Sorry, I'm a little confused with different implementations of multi-threading, like mpi, mpi+pthread, coarse, coarse+mpi and maybe coarse+mpi+pthread (?). I have seen improvements on coarse version on single node. I'm currently running mpiexec --host s1,s2 -np 2 raxml-ng-coarse --workers 8 --threads 32 --msa aln.fa --model LG+I+F+G4 --bs-trees 500 on two 32-core machines, is this rational? (version 27.10.2019)

amkozlov commented 4 years ago

Proper documentation on coarse-grained parallelization is coming soon...

Meanwhile, please read: https://github.com/amkozlov/raxml-ng/wiki/Parallelization and in particular: https://github.com/amkozlov/raxml-ng/wiki/Parallelization#coarse-grained-parallelization-for-short-alignments

This should give you an idea on how many workers to use.