hallamlab / metapathways2

MetaPathways v2.0: A master-worker model for environmental Pathway/Genome Database construction on grids and clouds
http://hallam.microbiology.ubc.ca/MetaPathways/
33 stars 14 forks source link

MP2: "BLAST Database error: Could not find volume or alias file" #82

Closed taltman closed 7 years ago

taltman commented 8 years ago

When Blast+ constructs a large index, it splits it into chunks, and they're all mapped together in the *.pal file.

This is fine, but MP2 tries to provide atomic database update by calling the database with a "temp" suffix, so that the database can be renamed "in place" once done. The problem here is that this works for small databases, but then when you have large databases, these pal files get written with the "temp" string in the chunk file names:

#
# Alias file created: Mar 11, 2016  7:17 AM
#
TITLE /home/altman/dbs/MetaPathways_DBs//functional/SEED_2013-06-19.fasta
DBLIST SEED_2013-06-19.fasta__temp__.00 SEED_2013-06-19.fasta__temp__.01 SEED_2013-06-19.fasta__temp__.02

When MP does the name replacement, it does it only on the files, and not within the files, and so you get errors like this when you try to use the database:

BLASTING AMINO SEQS AGAINST SEED_2013-06-19.fasta           ERROR       Cannot successfully execute the BLAST for FUNC_SEARCH
ERROR   BLAST Database error: Could not find volume or alias file (SEED_2013-06-19.fasta__temp__.00) referenced in alias file (/home/altman/dbs/MetaPathways_DBs/functional/formatted/SEED_20\
13-06-19.fasta).

The easy fix if you get this bug is to delete the "temp" string from the file names in the *.pal file.

If MP2 wants to retain this kind of feature, I'd recommend doing the following instead:

  1. Create a temporary directory
  2. Build the database, directing the program to put the index database files in the temporary directory
  3. Once complete, over-write the existing database files in the "formatted" directory with the ones just created in the temporary directory
  4. Test the database files to verify that they work
  5. Delete the temporary directory
hallamlab commented 7 years ago

This will be addressed in the newer version coming soon