search

metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
376 stars 98 forks source link

Error in rule root_tree #233

Closed jmtsuji closed 5 years ago

jmtsuji commented 5 years ago

I'm using ATLAS 2.1.4 on a Linux server (not a cluster) running Ubuntu 18.04.

After running the workaround stated in issue #231, I get a separate ImportError in rule root_tree:

Traceback (most recent call last):
  File "/Analysis/jmtsuji/Cfx3/.snakemake/scripts/tmparcfn_w4.tree.py", line 11, in <module>
    from . import parsers_checkm
ImportError: cannot import name 'parsers_checkm'
[Fri Sep 20 11:00:18 2019]
Error in rule root_tree:
    jobid: 3
    output: genomes/tree/tree.nwk
    conda-env: /Data/reference_databases/atlas/2.1.x/conda_envs/5bf853c6

    RuleException:
CalledProcessError in line 31 of /home/jmtsuji/miniconda3/envs/atlas_2.1.4/lib/python3.6/site-packages/atlas/rules/tree.smk:
Command 'source /home/jmtsuji/miniconda3/bin/activate '/Data/reference_databases/atlas/2.1.x/conda_envs/5bf853c6'; set -euo pipefail;  python /Analysis/jmtsuji/Cfx3/.snakemake/scripts/tmparcfn_w4.tree.py' returned non-zero exit status 1.
  File "/home/jmtsuji/miniconda3/envs/atlas_2.1.4/lib/python3.6/site-packages/atlas/rules/tree.smk", line 31, in __rule_root_tree
  File "/home/jmtsuji/miniconda3/envs/atlas_2.1.4/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Job failed, going on with independent jobs.

For some reason, the tree.py seems unable to load parsers_checkm in:

from . import parsers_checkm

Any insights here? Thanks.

tianlei22 commented 5 years ago

I met the same error.

jmtsuji commented 5 years ago

@SilasK Thanks for working on this. Did you notice my comment at https://github.com/metagenome-atlas/atlas/pull/234 ? I commented after the PR was merged, so it's possible you missed it.

jmtsuji commented 5 years ago

@SilasK I could make a PR for this to the master branch if helpful.

SilasK commented 5 years ago

e4c0069

Solved this, didn't it?

jmtsuji commented 5 years ago

@SilasK One additional bug at the end of the script:

phyla= parsers_checkm.load_checkm_tax(snakemake.input.taxonomy).phylum

Should be:

phyla= load_checkm_tax(snakemake.input.taxonomy).phylum

(because load_checkm_tax is now a function within the script itself)

SilasK commented 5 years ago

Thank you. On 27 Sep 2019, 05:19 +0200, metagenome-atlas/atlas reply@reply.github.com, wrote:

load_checkm_tax(snakemake.input.taxonomy).phylum

jmtsuji commented 5 years ago

@SilasK Thanks -- looks good.

Sofie8 commented 5 years ago

Hello Jackson, Silas,

I was using atlas 2.14, but I installed the dev version, 2.1.6 with the suggested fixes.

I included your suggestion Jackson: phyla= parsers_checkm.load_checkm_tax(snakemake.input.taxonomy).phylum to: phyla= load_checkm_tax(snakemake.input.taxonomy).phylum

However, I receive another error now:

[Thu Oct 10 03:02:02 2019] localrule root_tree: input: genomes/checkm/storage/tree/fasttree.nwk, genomes/checkm/taxonomy.tsv output: genomes/tree/tree.nwk jobid: 2

Activating conda environment: /ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc Traceback (most recent call last): File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/Shotgun/Francisco/.snakemake/scripts/tmp31y2a2av.tree.py", line 76, in phyla= load_checkm_tax(snakemake.input.taxonomy).phylum File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/Shotgun/Francisco/.snakemake/scripts/tmp31y2a2av.tree.py", line 28, in load_checkm_tax checkmTax.columns=['kindom','phylum','class','order','family','genus','species'] File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc/lib/python3.6/site-packages/pandas/core/generic.py", line 3094, in setattr return object.setattr(self, name, value) File "pandas/_libs/src/properties.pyx", line 65, in pandas._libs.lib.AxisProperty.set (pandas/_libs/lib.c:45255) File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc/lib/python3.6/site-packages/pandas/core/generic.py", line 473, in _set_axis self._data.set_axis(axis, labels) File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc/lib/python3.6/site-packages/pandas/core/internals.py", line 2836, in set_axis (old_len, new_len)) ValueError: Length mismatch: Expected axis has 5 elements, new values have 7 elements

Any solutions for this? Thank you so much.

SilasK commented 5 years ago

You have pretty unknown genomes :-). None of them had a species annotation. I have to check this.

You can change to the branch 'gtdb' see PR. It uses the GTDB for Taxonomy annotation and builds a tree.

On Thu, Oct 10, 2019, 03:32 Sofie8 notifications@github.com wrote:

Hello Jackson, Silas,

I was using atlas 2.14, but I installed the dev version, 2.1.6 with the suggested fixes.

I included your suggestion Jackson: phyla= parsers_checkm.load_checkm_tax(snakemake.input.taxonomy).phylum to: phyla= load_checkm_tax(snakemake.input.taxonomy).phylum

However, I receive another error now:

[Thu Oct 10 03:02:02 2019] localrule root_tree: input: genomes/checkm/storage/tree/fasttree.nwk, genomes/checkm/taxonomy.tsv output: genomes/tree/tree.nwk jobid: 2

Activating conda environment: /ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc Traceback (most recent call last): File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/Shotgun/Francisco/.snakemake/scripts/ tmp31y2a2av.tree.py", line 76, in phyla= load_checkm_tax(snakemake.input.taxonomy).phylum File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/Shotgun/Francisco/.snakemake/scripts/ tmp31y2a2av.tree.py", line 28, in load_checkm_tax

checkmTax.columns=['kindom','phylum','class','order','family','genus','species'] File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc/lib/python3.6/site-packages/pandas/core/generic.py", line 3094, in setattr return object.setattr(self, name, value) File "pandas/_libs/src/properties.pyx", line 65, in pandas._libs.lib.AxisProperty.set (pandas/_libs/lib.c:45255) File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc/lib/python3.6/site-packages/pandas/core/generic.py", line 473, in _set_axis self._data.set_axis(axis, labels) File "/ddn1/vol1/site_scratch/leuven/314/vsc31426/db/atlas2/conda_envs/f68a61bc/lib/python3.6/site-packages/pandas/core/internals.py", line 2836, in set_axis (old_len, new_len)) ValueError: Length mismatch: Expected axis has 5 elements, new values have 7 elements

Any solutions for this? Thank you so much.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/metagenome-atlas/atlas/issues/233?email_source=notifications&email_token=ABZMVWV7VG2CQCMUL2GDS3TQN2A4FA5CNFSM4IYYZXMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA2BJZA#issuecomment-540284132, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZMVWS3T3HU3TPCKVD5SUDQN2A4FANCNFSM4IYYZXMA .

Sofie8 commented 5 years ago

Switching to the GTDB branch indeed worked! Thanks!

On a side note, with the install of conda_env for gtdb and also das_tool, my cluster HPC system asks each time for, and exits: libstdcxx-ng, libgcc-ng

NoPackagesFoundError: Dependencies missing in current linux-64 channels:

How can I specify it in the yaml file, that it first downloads this libstdcxx and libgcc (anaconda) and then the gtdb or das_tool packages?

SilasK commented 5 years ago

Hey @Sofie8 cool to hear from you.

SilasK commented 5 years ago

@sofie8 If this is the explication we should all uptdate to conda 4.6

SilasK commented 5 years ago

Fixed by #257