search

antonisdim / haystac

Code repository for the HAYSTAC pipeline
MIT License
12 stars 4 forks source link

RuleException: ImportError in line 45 of /haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' #17

Closed Yaqiao-Li closed 11 months ago

Yaqiao-Li commented 1 year ago

Hi team,

I installed haystac following the instructions on github using mamba. I am running the program on AWS c5.9xlarge EC2 instance with 36cCPU 72GiB memory. Linux version 5.15.0-1004-aws (buildd@lcy02-amd64-093) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #6-Ubuntu SMP Thu Mar 31 09:44:20 UTC 2022

`conda install -c conda-forge mamba mamba create -c conda-forge -c bioconda -n haystac haystac conda activate haystac

haystac database \ --mode build \ --refseq-rep prokaryote_rep \ --output refseq_db ` The error message:

`HAYSTAC v 0.4.10

Date: 2023-05-09 17:57:38.023122

Config parameters:

mode: build db_output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db refseq_rep: prokaryote_rep bowtie2_scaling: 25.0 bowtie2_threads_db: 4 rank: species cores: 36 mem: 70227

Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 36 Rules claiming more threads will be scaled down. Provided resources: entrez_api=3, mem_mb=70227 Job counts: count jobs 1 calculate_db_chunks 1 download_refseq_representative_table 1 entrez_db_list 1 entrez_refseq_rep_prok_accessions 1 index_all_accessions 1 index_all_db_chunks 1 randomise_db_order 7

[Tue May 9 17:57:38 2023] Job 2: Downloading the list of representative species from RefSeq.

Activating conda environment: /home/ubuntu/haystac/cache/conda/db05c3cf54cbc90bcd1cd9bfd099d359 [Tue May 9 17:57:39 2023] Finished job 2. 1 of 7 steps (14%) done

[Tue May 9 17:57:39 2023] Job 1: Splitting the representative RefSeq table in smaller tables. Downstream jobs will be updated after completion.

[Tue May 9 17:57:39 2023] Error in rule entrez_refseq_rep_prok_accessions: jobid: 1 output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/assemblies.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-plasmids.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-plasmids.tsv log: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log (check log file(s) for error message)

RuleException: ImportError in line 45 of /home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 2352, in run_wrapper File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk", line 45, in rule_entrez_refseq_rep_prok_accessions File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/init.py", line 32, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/validator.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/json_compat.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/init.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/_format.py", line 7, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/exceptions.py", line 9, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/init.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/converters.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/_compat.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 569, in _callback File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/concurrent/futures/thread.py", line 56, in run File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 555, in cached_or_run File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init__.py", line 2364, in run_wrapper Trying to restart job 1.

[Tue May 9 17:57:39 2023] Job 1: Splitting the representative RefSeq table in smaller tables. Downstream jobs will be updated after completion.

[Tue May 9 17:57:39 2023] Error in rule entrez_refseq_rep_prok_accessions: jobid: 1 output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/assemblies.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-plasmids.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-plasmids.tsv log: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log (check log file(s) for error message)

RuleException: ImportError in line 45 of /home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 2352, in run_wrapper File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk", line 45, in rule_entrez_refseq_rep_prok_accessions File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/init.py", line 32, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/validator.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/json_compat.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/init.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/_format.py", line 7, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/exceptions.py", line 9, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/init.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/converters.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/_compat.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 569, in _callback File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/concurrent/futures/thread.py", line 56, in run File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 555, in cached_or_run File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init__.py", line 2364, in run_wrapper Trying to restart job 1.

[Tue May 9 17:57:39 2023] Job 1: Splitting the representative RefSeq table in smaller tables. Downstream jobs will be updated after completion.

[Tue May 9 17:57:39 2023] Error in rule entrez_refseq_rep_prok_accessions: jobid: 1 output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/assemblies.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-plasmids.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-plasmids.tsv log: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log (check log file(s) for error message)

RuleException: ImportError in line 45 of /home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 2352, in run_wrapper File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk", line 45, in rule_entrez_refseq_rep_prok_accessions File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/init.py", line 32, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/validator.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/json_compat.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/init.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/_format.py", line 7, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/exceptions.py", line 9, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/init.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/converters.py", line 10, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/_compat.py", line 12, in File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 569, in _callback File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/concurrent/futures/thread.py", line 56, in run File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 555, in cached_or_run File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init__.py", line 2364, in run_wrapper Job failed, going on with independent jobs. Exiting because a job execution failed. Look above for error message `

Could anyone please help me with this problem? Thank you very much!

antonisdim commented 1 year ago

Hello,

I hope you are doing great and apologies for not getting back to you sooner !

Unfortunately I have not been able to reproduce the error cannot import name '_GenericAlias'.

Would you possibly be able to post the contents of this log file /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log, in case there are any helpful leads in there ?

Of course in the meantime I'll keep looking into what could have caused the above error.

Thank you for your help and patience ! Antony