search

polio-nanopore / piranha

GNU General Public License v3.0
16 stars 4 forks source link

Resource allocation and HPC computing #90

Open DarianHole opened 1 year ago

DarianHole commented 1 year ago

Hi @aineniamh,

I was wondering if you have any information on the resources we should give Piranha to maximize its efficiency along with if you've tried to make use of snakemake's profiles feature on an HPC environment to speed up jobs? Or what other suggestions you may have regarding this?

I've been just playing around with export SNAKEMAKE_PROFILE=path/to/myprofile for the moment

Best, Darian

aineniamh commented 1 year ago

Hi @DarianHole, I haven't had a chance to play around with the most efficient use of piranha and so far have been testing locally rather than on an HPC. In the main pipeline, the snakemake can expand to run each barcode in parallel, and the side pipelines (within barcode analysis) can also be parallelised depending on how many threads specified at the beginning.

I'd be interested to hear if you find any useful regarding use in HPC or think of particular things that could help with the efficiency!