search

metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
376 stars 98 forks source link

Improvement of offline cluster mode #666

Closed jotech closed 1 year ago

jotech commented 1 year ago

Our slurm cluster only works in offline mode, so there is no possibility of downloading files when executing jobs. Although the atlas pipeline has a downloading step, some resources are not downloaded. Especially the rules download_gunc and dram_download fail when submitted as jobs by snakemake.

As an ad-hoc fix, I modified atlas/workflow/rules/download.smk by adding download_gunc to the localrules block. Similarly, I added a localrules block with dram_download for dram in atlas/workflow/rules/dram.smk. My question is, does this approach make sense, or are there other ways to do it?

Could it make sense to move the steps to atlas download? (potentially including busco_download, but I haven't encountered an error with it, yet.)

SilasK commented 1 year ago

I am hesitant to add every possible database to the download rule, maybe users don't want them but making them local is devenitively a good idea. Do you want to make a PR? Or shuld I implement the changes? download busco is not used for now.

jotech commented 1 year ago

great thank you for your feedback! I opened a PR