search

naturalis / barcode-constrained-phylogeny

Pipeline for building topologically-constrained phylogenies from DNA barcode data
https://naturalis.github.io/barcode-constrained-phylogeny/
Apache License 2.0
2 stars 3 forks source link

Scalability of large subsets #100

Open rvosa opened 5 months ago

rvosa commented 5 months ago

In the current BOLD database, nearly all taxonomic families are small enough to be tractable for the tree inference engine, raxml. The rule of thumb that is followed, is that raxml ought to be able to handle up to ±10k input sequences. Since the tree searches are topologically constrained, the overall search space is significantly reduced, meaning that this may indeed be a realistic rule. Nevertheless, a small number of families (<10) exceed this size. These families have not been attempted yet. As a first step, this needs to be tested. If there are problems, these families will need to be partitioned further, e.g. down to the subfamily or genus level. This will have implications for the parallelization strategy (see #99).

This issue is considered 'done' when the BOLD 10M data set has been processed successfully.

rvosa commented 4 months ago

@AnnemiekeSchonthaler, could you attach the plots you generated for the Lepidoptera, showing the running times?

AnnemiekeSchonthaler commented 4 months ago

@rvosa I attached the plots in the doc folder on the lepidoptera branch.