Parallel blast - Githubissues

VDBWRAIR / pathdiscov

Pathogen Discover Pipeline

1 stars 1 forks source link

Parallel blast #280

Open necrolyte2 opened 9 years ago

necrolyte2 commented 9 years ago

Waiting for VDBWRAIR/bio_pieces#46 to be completed and also next version to be released

necrolyte2 commented 9 years ago

I'm integrating changes from VDBWRAIR/bio_pieces#61 just to ensure parallel_blast is actually finally correct

necrolyte2 commented 9 years ago

TODO

[ ] Run pbs job to ensure it works
[ ] Compare results both from the diamond/blast results as well as the time taken

necrolyte2 commented 9 years ago

I'm running a test script that will run par_block_blast.pl and parallel_blast with the same parameters using blastx on a fasta file that has 304 entries.

I'm running it on my local computer with 4 instances and have a job queued to run on the cluster with 10 cpus

Hopefully this will show for sure if parallel_blast is faster or not(especially now that we are using --round-robin)

necrolyte2 commented 9 years ago

https://gist.github.com/necrolyte2/6ff2e4dab8183fd2a027

On my machine par_block and parallel_blast took almost the same amount of time.

parallel_blast 4 cpu - 06:08:00
par_block_blast.pl - 06:34:00

Running on our cluster with a single node and 10 cpu now and then will run parallel-blast with a few nodes to see how fast it speeds up

necrolyte2 commented 9 years ago

Tracking progress in this badly formatted google doc https://docs.google.com/spreadsheets/d/1D3VdTsJqBZ8zVpdkmj3VxWIqrX-UFCBTzehp5K2-FW0/edit?usp=sharing