search

Finesim97 / MetaErgInstallHelper

A few scripts to help with the installation of MetaErg, the contig/bin annotation pipeline
Academic Free License v3.0
6 stars 3 forks source link

cannot find arc.hmm database #5

Closed s4251484 closed 4 years ago

s4251484 commented 4 years ago

Hi @Finesim97 and team again

I have encountered an error with the testing set, the attach file is the verbose output metaerg_arc.hmm-error.txt

error: [09:50:09] Obtained from https://sourceforge.net/projects/rrnafinder/ [09:50:09] Can't find database: /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../db/hmm/rna/arc.hmm [Thu May 21 09:50:09 2020] Could not run command:/QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/rRNAFinder.pl --dbdir /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../db --threads 8 --evalue 1e-05 --domain meta --outdir metaergtest/tmp metaergtest/tmp/metaerg.pl_05212020.tRNA.masked.fna, No such file or directory

Finesim97 commented 4 years ago

Hi again, The specification of the database location is currently broken in MetaErg, I don’t think that is going to get fixed in the near future. Will update my Readme. You have to place the reference data in /QRISdata/Q2348/metaerg/metaerginstall/metaerg/db

Hope this helps, Lukas

Am 21.05.2020 um 02:05 schrieb s4251484 notifications@github.com:

Hi @Finesim97 and team again

I have encountered an error with the testing set, the attach file is the verbose output metaerg_arc.hmm-error.txt

error: [09:50:09] Obtained from https://sourceforge.net/projects/rrnafinder/ [09:50:09] Can't find database: /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../db/hmm/rna/arc.hmm [Thu May 21 09:50:09 2020] Could not run command:/QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/rRNAFinder.pl --dbdir /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../db --threads 8 --evalue 1e-05 --domain meta --outdir metaergtest/tmp metaergtest/tmp/metaerg.pl_05212020.tRNA.masked.fna, No such file or directory

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

s4251484 commented 4 years ago

Hi again, The specification of the database location is currently broken in MetaErg, I don’t think that is going to get fixed in the near future. Will update my Readme. You have to place the reference data in /QRISdata/Q2348/metaerg/metaerginstall/metaerg/db Hope this helps, Lukas … Am 21.05.2020 um 02:05 schrieb s4251484 @.***>: Hi @Finesim97 and team again I have encountered an error with the testing set, the attach file is the verbose output metaerg_arc.hmm-error.txt error: [09:50:09] Obtained from https://sourceforge.net/projects/rrnafinder/ [09:50:09] Can't find database: /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../db/hmm/rna/arc.hmm [Thu May 21 09:50:09 2020] Could not run command:/QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/rRNAFinder.pl --dbdir /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../db --threads 8 --evalue 1e-05 --domain meta --outdir metaergtest/tmp metaergtest/tmp/metaerg.pl_05212020.tRNA.masked.fna, No such file or directory — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

i see. thanks for your reply! but what do you mean by reference data? will wait on the updated readme! thanks again!

Finesim97 commented 4 years ago

Did you download http://ebg.ucalgary.ca/metaerg/db.tar.gz or did you use setup_db.pl?

s4251484 commented 4 years ago

Did you download http://ebg.ucalgary.ca/metaerg/db.tar.gz or did you use setup_db.pl?

i downloaded from http://ebg.ucalgary.ca/metaerg/db.tar.gz

Finesim97 commented 4 years ago

Could you send me the output of:

ls /QRISdata/Q2348/metaerg/metaerginstall/metaerg/
ls /QRISdata/Q2348/metaerg/metaerginstall/metaerg/db
s4251484 commented 4 years ago

Could you send me the output of:

ls /QRISdata/Q2348/metaerg/metaerginstall/metaerg/
ls /QRISdata/Q2348/metaerg/metaerginstall/metaerg/db

`s4251484@tn513d:~> ls /QRISdata/Q2348/metaerginstall/metaerg/

bin db Dockerfile docs example hmmrna metaerg.sql notes.txt README.md template tmp txt

s4251484@tn513d:~> ls /QRISdata/Q2348/metaerginstall/metaerg/db

blast diamond hmm sqlite3`

Sure! i re-installed metaerg following your read.me so the path has changed.

Finesim97 commented 4 years ago

Ok, seems fine and there is no /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm ?

s4251484 commented 4 years ago

Ok, seems fine and there is no /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm ?

yea (/QRISdata/Q2348/metaerginstall) s4251484@tn513d:~> find /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm find: ‘/QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm’: No such file or directory (/QRISdata/Q2348/metaerginstall) s4251484@tn513d:~> ls /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm ls: cannot access /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm: No such file or directory (/QRISdata/Q2348/metaerginstall) s4251484@tn513d:~> less /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna/arc.hmm: No such file or directory

is it possible that it is a temporary file? coz I don't see any db folder in /QRISdata/Q2348/metaerginstall/metaerg/bin

Finesim97 commented 4 years ago

That is strange.

The .. in /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../dbmeans "go up one directory", therefore the used path is /QRISdata/Q2348/metaerg/metaerginstall/metaerg/db.

Sry for all the lss, but what is in the folder /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/ and /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna

s4251484 commented 4 years ago

That is strange.

The .. in /QRISdata/Q2348/metaerg/metaerginstall/metaerg/bin/../dbmeans "go up one directory", therefore the used path is /QRISdata/Q2348/metaerg/metaerginstall/metaerg/db.

Sry for all the lss, but what is in the folder /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/ and /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna

(/RDS/Q1034/conda/taxmaps) s4251484@tn513d:~> ls /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/

casgenes.hmm FOAM1.hmm FOAM2.hmm FOAM3.hmm FOAM4.hmm FOAM5.hmm FOAM-hmm_rel1a.hmm metabolic.hmm Pfam-A.hmm TIGRFAMs.hmm

(/RDS/Q1034/conda/taxmaps) s4251484@tn513d:~> /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna

-bash: /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna: No such file or directory`

the db.tar.gz was 20GB No worries! i would be very glad if this can be solved. and I have a feeling you are getting there! thanks for your help!

Finesim97 commented 4 years ago

Ok, my installation also doesn't have this folder, but I am using an older version, in my version, the hmmrna folder in metaerg is used, which has the missing files and comes with the metaerg installation. The setup_db.pl of the current version creates these files at the correct path, but the reference data provided by the authors is missing this folder.

The solution therefore is:

cp -r   /QRISdata/Q2348/metaerginstall/metaerg/hmmrna /QRISdata/Q2348/metaerginstall/metaerg/db/hmm/rna
s4251484 commented 4 years ago

IT IS A FIX!!! THANKS! run finished successfully with the testing set!

MatteoSchiavinato commented 3 years ago

Did you download http://ebg.ucalgary.ca/metaerg/db.tar.gz or did you use setup_db.pl?

i downloaded from http://ebg.ucalgary.ca/metaerg/db.tar.gz

Hi, I came onto this fix but this link doesn't work anymore (404). Could you provide a new one?

Finesim97 commented 3 years ago

Did you download http://ebg.ucalgary.ca/metaerg/db.tar.gz or did you use setup_db.pl?

i downloaded from http://ebg.ucalgary.ca/metaerg/db.tar.gz

Hi, I came onto this fix but this link doesn't work anymore (404). Could you provide a new one?

Hi, I am not sure, why they removed the database, either because it was to big and downloaders used too much bandwidth or (wthat I fear is more likely) they were redistributing reference data, which they weren't allowed to redistribute (some databases, especially commercial ones have very strict policies). So I am not ready to upload them myself to my own server. Do you have the computational resources to create the databases files yourself?

Have a nice, day, Lukas Jansen

MatteoSchiavinato commented 3 years ago

Hi, thanks for the honest answer. Using the setup script only downloaded and built a subset of the datasets. I could build them myself, that's for sure: are there instructions to do it somewhere, or do I find them on the corresponding websites?

Finesim97 commented 3 years ago

The script in this repository should (haven't tested it recently) install the dependencies for making the setup_db.pl call run completely, this should lead to a full database. I haven't tested the changes pushed to the MetaErg repository in January of this year. Honestly you should maybe consider running other, better supported pipelines, I used metaErg in my bachelor thesis and project, but in the lab we called it MetaÄrger (German for trouble). I can recommend sourmash (for tax. analysis), interproscan, kofamkoala (there is a version, which you can run on your own system) and eggNOG mapper (those 3 for functional analysis). Hope you find something, that works for you Lukas Jansen