search

bio-miga / miga

MiGA: Microbial Genomes Atlas
Artistic License 2.0
31 stars 9 forks source link

Part of MiGA is not working (aai and ani) #124

Closed BJCampbelllab closed 3 years ago

BJCampbelllab commented 3 years ago

Hi MiGA team! I installed MiGA via conda, and followed the installation instructions. Got it to work for taxonomy calling. However, when I go to use miga wf, for index (aai, ani), it gives me this error (listed below in miga.project.log) - even with three genomes. It seems to go through all the steps until then. For example, the log of one of the genomes is below.

Any help would be greatly appreciated! I would do it on your website, but I have more than 50 genomes!

Best, Barb

daemon/d log of one of the genomes:

[ cds ]

Date: 2021-02-24 10:06:27 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

Dataset: GCA_000964845

------------

Selected codon table: 11

[ essential_genes ]

Date: 2021-02-24 10:06:41 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

Dataset: GCA_000964845

------------

Temporal directory: /tmp/d20210224-47452-94wovf. Searching models. Parsing results. Extracting sequences. Done.

[ ssu ]

Date: 2021-02-24 10:06:50 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

Dataset: GCA_000964845

------------

index file ../../../05.assembly/GCA_000964845.LargeContigs.fna.fai not found, generating... Reading list. Filtering FastA.

[ mytaxa_scan ]

Date: 2021-02-24 10:06:56 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

Dataset: GCA_000964845

------------

[ distances ]

Date: 2021-02-24 10:07:00 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

Dataset: GCA_000964845

------------

Completeness: 93.7 % Contamination: 1.8 % Quality: 84.7 Options: {:aai_save_rbm=>"no-save-rbm", :thr=>1, :haai_p=>nil, :aai_p=>"diamond", :ani_p=>"fastani", :distances_checkpoint=>10} Launching analysis Launching analysis for reference dataset Initializing databases (for_ref = true) [ 2021-02-24 10:07:00 -0500 ] GCA_000011945 Accessing SQLite3 file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.haai.db. Temporal directory: /tmp/d20210224-5878-4j4yfq. Creating databases. Reading FastA file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.ess_genes.fa File contains 104 sequences. Reading FastA file: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test/data/07.annotation/01.function/01.essential/GCA_000011945.ess.faa File contains 102 sequences. Running one-way comparisons. Warning: [blastp] Examining 5 or more matches is recommended Warning: [blastp] Examining 5 or more matches is recommended [ 2021-02-24 10:07:04 -0500 ] GCA_000016625 Accessing SQLite3 file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.haai.db. Temporal directory: /tmp/d20210224-6323-1r3mnek. Creating databases. Reading FastA file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.ess_genes.fa File contains 104 sequences. Reading FastA file: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test/data/07.annotation/01.function/01.essential/GCA_000016625.ess.faa File contains 105 sequences. Running one-way comparisons. Warning: [blastp] Examining 5 or more matches is recommended Warning: [blastp] Examining 5 or more matches is recommended Accessing SQLite3 file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.aai.db. Temporal directory: /tmp/d20210224-8200-ynnrh4. Creating databases. Reading FastA file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.proteins.fa File contains 2492 sequences. Reading FastA file: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test/data/06.cds/GCA_000016625.faa.gz File contains 1579 sequences. Running one-way comparisons. [ 2021-02-24 10:07:19 -0500 ] GCA_000964845 Accessing SQLite3 file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.haai.db. Temporal directory: /tmp/d20210224-10587-1lvtq0h. Creating databases. Reading FastA file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.ess_genes.fa File contains 104 sequences. Reading FastA file: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test/data/07.annotation/01.function/01.essential/GCA_000964845.ess.faa File contains 104 sequences. Running one-way comparisons. Warning: [blastp] Examining 5 or more matches is recommended Warning: [blastp] Examining 5 or more matches is recommended Accessing SQLite3 file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.aai.db. Temporal directory: /tmp/d20210224-11832-167fnvz. Creating databases. Reading FastA file: /tmp/d20210224-5877-1wg61rv/GCA_000964845.proteins.fa File contains 2492 sequences. Reading FastA file: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test/data/06.cds/GCA_000964845.faa.gz File contains 2492 sequences. Running one-way comparisons. Checkpoint (metric = haai) Checkpoint (metric = aai) Checkpoint (metric = ani)

[ stats ]

Date: 2021-02-24 10:08:13 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

Dataset: GCA_000964845

------------

raw_reads

trimmed_fasta

assembly

Contigs: 49 N50: 347385 bp Total length: 2303093 bp Longest sequence: 1137055 bp G+C content: 32.39830089362436 % X content: 0.0 % G-C skew: 1.9763778155711285 % A-T skew: 0.6056791249446025 %

cds

Predicted proteins: 2492 Average length: 253.27969502407706 aa Coding density: 82.21634992594741 % Codon table: 11

essential_genes

Completeness: 93.7 % Contamination: 1.8 % Quality: 84.7

ssu

Ssu: 1 Complete ssu: 1 Max length: 1494

distances

taxonomy

Last error in miga-project.log (keeps doing over and over again...):

[ clade_finding ]

Date: 2021-02-24 20:23:59 -0500

Host: localhost.localdomain

MiGA: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1

Project: /home/bcampbell/Pelagibacterales/Pelagibacter_for_v7/original_fasta/Rickettsia/test

------------

[ Wed Feb 24 20:24:02 2021 ] ==> Out base: miga-project <== Exception: Cannot add result, incomplete expected files.

DEBUG: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action/add_result.rb:23:in perform' DEBUG: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action.rb:32:inlaunch' DEBUG: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli.rb:177:in launch' DEBUG: /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/bin/miga:12:in

' /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action/add_result.rb:23:in perform': Cannot add result, incomplete expected files. (RuntimeError) from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action.rb:32:inlaunch' from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli.rb:177:in launch' from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/bin/miga:12:in
' /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action/run.rb:77:in wait': Interrupt from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action/run.rb:77:inperform' from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli/action.rb:32:in launch' from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/lib/miga/cli.rb:177:inlaunch' from /home/bcampbell/anaconda3/envs/miga/share/rubygems/gems/miga-base-0.7.20.1/bin/miga:12:in `
'

Generates a MiGA.output file (haven't attempted to open it).

jianshu93 commented 3 years ago

Dear Barb,

I am using the Conda version following the installation here: https://colab.research.google.com/drive/1ybCCPbbZtJ41HC_1yNYed8Yf-q9CDY5a

the index_wf works nicely for me and I have the Rdata for AAI and ANI. Luis Miguel may have other suggestions for you because he's working on a new release I think.

Thank you for your interest in MiGA.

Jianshu

lmrodriguezr commented 3 years ago

Hello @BJCampbelllab I believe this is related to a short-lived bug that was introduced in v0.7.20.0 but has now been resolved. Please update MiGA to the latest version (currently v0.7.25.3) and try again. Please let us know if you continue to experience this issue or if you encounter a new problem.

Thank you for your report! Miguel.

lmrodriguezr commented 3 years ago

I'm closing this issue, but please feel free to reopen and/or just comment here.

BJCampbelllab commented 3 years ago

Thanks! I’ll update and see what happens.

From: Luis M Rodriguez-R notifications@github.com Sent: Monday, March 1, 2021 8:08 AM To: bio-miga/miga miga@noreply.github.com Cc: Barbara Campbell bcampb7@clemson.edu; Mention mention@noreply.github.com Subject: Re: [bio-miga/miga] Part of MiGA is not working (aai and ani) (#124)

I'm closing this issue, but please feel free to reopen and/or just comment here.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/bio-miga/miga/issues/124#issuecomment-787935968, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG6TSRMFG7RJZ4KDAOPK5ZDTBOGUVANCNFSM4YLIV6HQ.

BJCampbelllab commented 3 years ago

Hi Luis, I ended up removing the miga environment from conda by rm the environment/directory and am attempting to reinstall from these instructions: https://manual.microbial-genomes.org/part2/requirements/conda

I got an error for scythe. It was no longer found in faircloth-lab.

PackagesNotFoundError: The following packages are not available from current channels:

Current channels:

I went to the anaconda page and searched for scythe, and that only gave me access to a scythe-score-counter (tpug).

Any suggestions? Thanks! Barb

From: Luis M Rodriguez-R notifications@github.com Sent: Monday, March 1, 2021 8:06 AM To: bio-miga/miga miga@noreply.github.com Cc: Barbara Campbell bcampb7@clemson.edu; Mention mention@noreply.github.com Subject: Re: [bio-miga/miga] Part of MiGA is not working (aai and ani) (#124)

Hello @BJCampbelllabhttps://github.com/BJCampbelllab I believe this is related to a short-lived bug that was introduced in v0.7.20.0 but has now been resolved. Please update MiGA to the latest version (currently v0.7.25.3) and try again. Please let us know if you continue to experience this issue or if you encounter a new problem.

Thank you for your report! Miguel.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/bio-miga/miga/issues/124#issuecomment-787934623, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG6TSRIZH33B36IWXCLMOOTTBOGM5ANCNFSM4YLIV6HQ.