search

epi2me-labs / wf-metagenomics

Metagenomic classification of long-read sequencing data
Other
56 stars 23 forks source link

ELPI2ME - DOCKER - Error #62

Closed EsbergA closed 11 months ago

EsbergA commented 1 year ago

Ask away!

I hope that someone can help me with this issue, I´m trying to convert HOMD-ref-seq and HOMD-taxa (https://www.homd.org/download#refseq) to be used with wf-metagenomics on my MAC Studio, but I frequently get two different errors:

  1. NOTE: Process kraken_pipeline:kraken_server terminated with an error exit status (65) -- Error is ignored

  2. Error executing process > 'kraken_pipeline:stop_kraken_server' Caused by: Process kraken_pipeline:stop_kraken_server terminated with an error exit status (14) Command executed: kraken2_client --port 8080 --shutdown Command exit status: 14 Command output: (empty) Command error: WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested Connecting to server: localhost:8080. Failed to send shutdown request. Shutdown request not processed correctly. Return code: 14

Any ideas on how to proceed as these errors make the analysis stall or terminate with error Best regards Anders

nggvs commented 1 year ago

Hi,

Thank you for using the workflow! As far as I understand, you're trying to use your own database (you can take a look on this). Could you send me the command you're running and the files of the database folder?

EsbergA commented 1 year ago

Dear Natalia, Thank you for a rapid response! I’m running wf-metagnomic on the EPI2MELab platform and the pipeline/platform gives me this code after selecting:

{

Best Regards Anders

Från: Natalia Garcia Garcia @.> Datum: torsdag, 14 september 2023 14:00 Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>, Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] ELPI2ME - DOCKER - Error (Issue #62)

Hi,

Thank you for using the workflow! As far as I understand, you're trying to use your own database. Could you send me the command you're running and the files of the database folder?

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/62#issuecomment-1719314958, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSGDPX477GKENOFNJ4LX2LWXNANCNFSM6AAAAAA4X7K4YE. You are receiving this because you authored the thread.Message ID: @.***>

nggvs commented 1 year ago

Thank you for using the EPI2ME platform! If you're using the latest version of the app, could you use the report_issue button and sent me the tar.gz file that is generated (you can access to it using the activity log and going to the instance that is failing)? I can send you a pic if you need it.

I have the suspect that the problem is with the databases. Does your database folder contain the *.k2d files needed for kraken2 and the taxonomy one *.dmp files?

EsbergA commented 1 year ago

Dear Natalia

I´m currently trying to map full 16S genes against the HOMD database (also 16S based) so not my own rather one of the best oral microbiota database out there.

As I see it within the EPI2ME (Labs V4.1.1) platform I need (and tried) to add a HOMD-Ref-seq and HOMD-taxa file, but always stopped. I the report_issue is placed as a html in V4.1.1.1 and no tar.gz is available, try to link it here file:///Users/andersesberg/epi2melabs/instances/wf-metagenomics_3bfd4ea5-601b-4c9f-a4ed-13d4cdc02b29/output/execution/report.html.

/ Anders

Från: Natalia Garcia Garcia @.> Datum: torsdag, 14 september 2023 14:24 Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>, Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] ELPI2ME - DOCKER - Error (Issue #62)

Thank your for using the EPI2ME platform! If you're using the latest version of the app, could you use the report_issue button and sent me the tar.gz file that is generated?

I have the suspect that the problem is with the databases. Does your database folder contain the .k2d files needed for kraken2 and the taxonomy one .dmp files?

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/62#issuecomment-1719348453, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSH5SNQ4SELTDKEH5K3X2LZPHANCNFSM6AAAAAA4X7K4YE. You are receiving this because you authored the thread.Message ID: @.***>

nggvs commented 1 year ago

Ok!

First of all, I suggest you to update to the newest version of the app which is the 5.1.2! You can download it from here.

So, which are your input files for the database? Are you using directly the FASTA file downloaded from here: (https://www.homd.org/download#refseq) or you have built a kraken2database using it?

EsbergA commented 1 year ago

Hi, I’m using the FASTA file and the taxonomy files and compress them to tar.gz and zip, wrong? I will update my version, the DOCKER, just stopped for me as well. Anders

Skickat från Outlook för iOShttps://aka.ms/o0ukef

Från: Natalia Garcia Garcia @.> Skickat: Thursday, September 14, 2023 2:59:32 PM Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>; Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] ELPI2ME - DOCKER - Error (Issue #62)

Ok!

First of all, I suggest you to update to the newest version of the app which is the 5.1.2! You can download it from herehttps://labs.epi2me.io/downloads/.

So, which are your input files for the database? Are you using directly the FASTA file downloaded from here: (https://www.homd.org/download#refseq) or you have built a kraken2database using it?

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/62#issuecomment-1719406282, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSGFDGHWA6ZK2JGIFNDX2L5TJANCNFSM6AAAAAA4X7K4YE. You are receiving this because you authored the thread.Message ID: @.***>

nggvs commented 1 year ago

Well, so we've found what is causing the problem, but first things first!

Regarding the latest version of the App, have you been able to run any workflow on it? I would suggest to run the wf-alignment because it's faster to see if everything works fine. If there is an error, please copy the issue and send to me the file created after using the report_issue button.

The kraken2 pipeline uses a specific database which is not the fasta file. As this moment, the wf doesn't include the generation of the kraken2-database from a FASTA file.

First, you need to create the kraken2 database and after that the bracken files. Here they explain how to do it:

From both you should have a directory with the next files:

The content of each folder (if you use the default database) should be:

ls db_taxonomy_dir/
citations.dmp  delnodes.dmp  division.dmp  gc.prt  gencode.dmp  merged.dmp  names.dmp  nodes.dmp  readme.txt 

ls db_database_dir/
database1000mers.kmer_distrib  hash.k2d  opts.k2d  taxo.k2d

And then you can run the pipeline setting the paths of:

Let me know if this works for you!

EsbergA commented 1 year ago

Dear Natalia

After installing the new EPI2ME platform which is apparently is under construction for Apple M1 processes “coming soon” and not fully operational, docker stopped “Engine stopped” and now I cannot get docker to start again.

Disregarding this, I think that we have another problem that been recognized previously by other with regards to the error I posted previoulsly “image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8)”.

As when I run the older EPI2ME platform with 16S-18S NCBI library I got the same error as I got for the custom library. So my question, is there a functional EPI2ME wf-metagenomic – Docker pipeline for M1 processors?

(Docker Linux linux/amd64 - linux/arm64/v8 errors reported)

/ Anders

Från: Natalia Garcia Garcia @.> Datum: torsdag, 14 september 2023 16:20 Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>, Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] ELPI2ME - DOCKER - Error (Issue #62)

Well, so we've found what is causing the problem, but first things first!

Regarding the latest version of the App, have you been able to run any workflow on it? I would suggest to run the wf-alignment because it's faster to see if everything works fine. If there is an error, please copy the issue and send to me the file created after using the report_issue button.

The kraken2 pipeline uses a specific database which is not the fasta file. As this moment, the wf doesn't include the generation of the kraken2-database from a FASTA file. I'm going to attach below the instructions to create this database.

First, you need to create the kraken2 database and after that the bracken files. Here they explain how to do it:

From both you should have a directory with the next files:

The content of each folder (if you use the default database) should be:

ls db_taxonomy_dir/

citations.dmp delnodes.dmp division.dmp gc.prt gencode.dmp merged.dmp names.dmp nodes.dmp readme.txt

ls db_database_dir/

database1000mers.kmer_distrib hash.k2d opts.k2d taxo.k2d

And then you can run the pipeline setting the paths of:

Let me know if this works for you!

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/62#issuecomment-1719550239, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSGOSODTH3ROYEVK3ODX2MHDDANCNFSM6AAAAAA4X7K4YE. You are receiving this because you authored the thread.Message ID: @.***>

nggvs commented 1 year ago

Hi! Which version of the wf-metagenomics are you using? We have updated the wf (from v2.4.4) to run in ARM processors. Regarding the app, I will come to you as soon as possible. Meanwhile, just to be sure, could you confirm that you were following these instructions. Does it give to you any error that you can send to us? Which version of docker do you have installed?

EsbergA commented 1 year ago

Dear Natalia Thanks for all your help, I got the wf-metagenomics to run against the 16-ref-seq database after updating all the packages. I have a question regarding the downstream analysis after wf-metagenomics, I can see the number of reads and species assigned, but where do I see the % match so I can evaluate the match? I would like to set a cut-off of 95%, 97%, and 99% to evaluate the capacity of Nanopores sequencing characteristics to correctly define my mock sample. Best regards Anders

Från: Natalia Garcia Garcia @.> Datum: fredag, 15 september 2023 12:38 Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>, Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] ELPI2ME - DOCKER - Error (Issue #62)

Hi! Which version of the wf-metagenomics are you using? We have updated the wf (from v2.4.4) to run in ARM processors. Regarding the app, I will come to you as soon as possible. Meanwhile, just to be sure, could you confirm that you were following these instructionshttps://labs.epi2me.io/installation/#installation-on-macos. Does it give to you any error that you can send to us? Which version of docker do you have installed?

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/62#issuecomment-1721055925, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSEIQ7FAKSX6GBPCWLLX2QV2DANCNFSM6AAAAAA4X7K4YE. You are receiving this because you authored the thread.Message ID: @.***>

nggvs commented 1 year ago

Hi @EsbergA !

Glad to hear that everything is working fine! Would you mind to open this as a new issue question as it is not related with the original question? It is helpful to follow better the topic of each issue. Besides, if you consider that this specific issue has already been solved, could you close this one? Thank you in advance,

nggvs commented 1 year ago

Hi @EsbergA ! If you consider that this specific issue has already been solved, could you close it. Otherwise we will close it assuming it has been already solved.