metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
368 stars 97 forks source link

decontamination databases may overwrite each other #24

Closed SilasK closed 6 years ago

SilasK commented 6 years ago

I got the following exception during decontamination. F33_decontamination.log

A similar error was discussed here:

This means that you were running multiple different indexing processes in the same directory at the same time. Unless you use a different directory for each process, or specify a different index location with "path=", or specify a different build number, the indexes can overwrite each other leading to corrupt zip files (which, fortunately, normally get detected, as in this case).

If you want to do all of these mapping operations to the same references, just index once, wait for it to finish, and then run all the mapping operations without specifying "ref=". E.g.


bbsplit.sh ref=ecoli.fa,salmonella.fa

(wait for finish)

bbsplit.sh a.fq basename=outa%.fq bbsplit.sh b.fq basename=outb%.fq bbsplit.sh c.fq basename=outc%.fq ...etc


>If each one needs different references, then either run them serially, or use a different directory/build each time.

I definitively think that we should 
- make a rule for building the reference once for all decontamination steps. 
- use database file as input for decontamination step
- execute decontamination in a shadow directory
SilasK commented 6 years ago

introduced a rule which creates the database for all #27