Closed xapple closed 7 years ago
Hey,
That's no good. So the
Are you compiling the database from the original sequence or using the precompiled database from the NCBI's ftp (ftp://ftp.ncbi.nlm.nih.gov/blast/db)? I'm not sure if a compiled database was dropped a precompiled database in there it would pickup the files. Since this is the main way people will get their sequences we should update to the code to regenerate the names.txt file if not present.
To get the pipeline to regenerate file you'll have to get the original sequence from that compiled refseq database. You can get the fasta sequences out of a compiled blast database with the blastdbcmd in the blast+ package. It would be something like:
$ blastdbcmd -db refseq -dbtype prot -outfmt %f -out refseq_sequences.fasta
You can put the resulting refseq.fasta command in the parameter file and run again. It should generate a
Since doing the above is a bit of work, I'll provide you with my refseq names file to see if that fixes the problem in the interim.
Let me know how it goes.
I downloaded the files 1 to 8 from ftp://ftp.ncbi.nlm.nih.gov/blast/db/refseq_protein.0*.tar.gz as per the instructions on the documentation at "4. BLAST databases". You should probably update the doc to reflect the -names.txt file issue.
Then I used the fastacmd -D 1 -d refseq_protein -o plaintext.fasta
command provided in the blast package (not +) to extract the sequences as there doesn't seem to be any corresponding file available on the NCBI FTP website in ftp://ftp.ncbi.nlm.nih.gov/blast/db/FASTA. It produced the file I gave an excerpt of in my previous post.
If you think we just need to get the headers from that plaintext.fasta
in the refseq_protein-names.txt
file then a simple sed
command should suffice. Or is the process more complicated ? Do you need my e-mail to send me your version of the file ?
Try your sed command to extract all the header lines from that fasta. If you have no luck, ping me again with your email and I'll get you my file. If all else fails point the config to the new plaintext.fasta file (may want to rename it to something more refseq-like) and MetaPathways will reformat and do the whole thing again.
I'm almost finished the new documentation. My apologies on the delay, there's a lot going on around here.
Niels
OK I will try that ! No prob, I know that a developers time is always one of the most required resources in this world : )
We will continue trying to get this working as it would be nice to be able to analyze our metagenomes samples using this pipeline. The next problem is probably going to be parallelizing the BLAST jobs as we don't have a SUN cluster but instead a SLURM one.
Just wanted to mention here that i ran into an issue similar to xapple's and his fastacmd
solution seems to have worked for me. I had to leave the refseq_protein fasta file in the blastdb.
Please use the newest version of MP on GitHub
Please use the newest version of MP on GitHub
As I tried to add the
refseq
database to the metapathways pipeline, I am getting an additional error which I am uncertain how to solve. Here is the transcript:Apparently the
-names.txt
file contains only the headers of the sequences present in a given database. However, therefseq
database is provided directly in BLAST binary format from NCBI. We could use thefastacmd
software to reverse this and generate the plaintext version. Subsequently extracting only the headers, but I'm not sure that would be a good way to proceed. The result would contain several annotations per sequence: