Closed Yaqiao-Li closed 11 months ago
Hello,
I hope you are doing great and apologies for not getting back to you sooner !
Unfortunately I have not been able to reproduce the error cannot import name '_GenericAlias'
.
Would you possibly be able to post the contents of this log file /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log
, in case there are any helpful leads in there ?
Of course in the meantime I'll keep looking into what could have caused the above error.
Thank you for your help and patience ! Antony
Hi team,
I installed haystac following the instructions on github using mamba. I am running the program on AWS c5.9xlarge EC2 instance with 36cCPU 72GiB memory. Linux version 5.15.0-1004-aws (buildd@lcy02-amd64-093) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #6-Ubuntu SMP Thu Mar 31 09:44:20 UTC 2022
`conda install -c conda-forge mamba mamba create -c conda-forge -c bioconda -n haystac haystac conda activate haystac
haystac database \ --mode build \ --refseq-rep prokaryote_rep \ --output refseq_db ` The error message:
`HAYSTAC v 0.4.10
Date: 2023-05-09 17:57:38.023122
Config parameters:
mode: build db_output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db refseq_rep: prokaryote_rep bowtie2_scaling: 25.0 bowtie2_threads_db: 4 rank: species cores: 36 mem: 70227
Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 36 Rules claiming more threads will be scaled down. Provided resources: entrez_api=3, mem_mb=70227 Job counts: count jobs 1 calculate_db_chunks 1 download_refseq_representative_table 1 entrez_db_list 1 entrez_refseq_rep_prok_accessions 1 index_all_accessions 1 index_all_db_chunks 1 randomise_db_order 7
[Tue May 9 17:57:38 2023] Job 2: Downloading the list of representative species from RefSeq.
Activating conda environment: /home/ubuntu/haystac/cache/conda/db05c3cf54cbc90bcd1cd9bfd099d359 [Tue May 9 17:57:39 2023] Finished job 2. 1 of 7 steps (14%) done
[Tue May 9 17:57:39 2023] Job 1: Splitting the representative RefSeq table in smaller tables. Downstream jobs will be updated after completion.
[Tue May 9 17:57:39 2023] Error in rule entrez_refseq_rep_prok_accessions: jobid: 1 output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/assemblies.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-plasmids.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-plasmids.tsv log: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log (check log file(s) for error message)
RuleException: ImportError in line 45 of /home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 2352, in run_wrapper File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk", line 45, in rule_entrez_refseq_rep_prok_accessions File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/init.py", line 32, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/validator.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/json_compat.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/ init.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/_format.py", line 7, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/exceptions.py", line 9, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/ init.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/converters.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/_compat.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/ init.py", line 569, in _callback
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/concurrent/futures/thread.py", line 56, in run
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 555, in cached_or_run
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init__.py", line 2364, in run_wrapper
Trying to restart job 1.
[Tue May 9 17:57:39 2023] Job 1: Splitting the representative RefSeq table in smaller tables. Downstream jobs will be updated after completion.
[Tue May 9 17:57:39 2023] Error in rule entrez_refseq_rep_prok_accessions: jobid: 1 output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/assemblies.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-plasmids.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-plasmids.tsv log: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log (check log file(s) for error message)
RuleException: ImportError in line 45 of /home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 2352, in run_wrapper File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk", line 45, in rule_entrez_refseq_rep_prok_accessions File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/init.py", line 32, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/validator.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/json_compat.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/ init.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/_format.py", line 7, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/exceptions.py", line 9, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/ init.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/converters.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/_compat.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/ init.py", line 569, in _callback
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/concurrent/futures/thread.py", line 56, in run
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 555, in cached_or_run
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init__.py", line 2364, in run_wrapper
Trying to restart job 1.
[Tue May 9 17:57:39 2023] Job 1: Splitting the representative RefSeq table in smaller tables. Downstream jobs will be updated after completion.
[Tue May 9 17:57:39 2023] Error in rule entrez_refseq_rep_prok_accessions: jobid: 1 output: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-genomes.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/assemblies.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-plasmids.tsv, /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/genbank-plasmids.tsv log: /home/ubuntu/17.20230410-Metagenomics-Larry/06.classification/02.haystac/refseq_db/entrez/refseq-rep-seqs.log (check log file(s) for error message)
RuleException: ImportError in line 45 of /home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk: cannot import name '_GenericAlias' File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 2352, in run_wrapper File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/haystac/workflow/rules/refseq.smk", line 45, in rule_entrez_refseq_rep_prok_accessions File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/init.py", line 32, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/validator.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/nbformat/json_compat.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/ init.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/_format.py", line 7, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/jsonschema/exceptions.py", line 9, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/ init.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/converters.py", line 10, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/attr/_compat.py", line 12, in
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/ init.py", line 569, in _callback
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/concurrent/futures/thread.py", line 56, in run
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init.py", line 555, in cached_or_run
File "/home/ubuntu/miniconda3/envs/haystac_3/lib/python3.6/site-packages/snakemake/executors/init__.py", line 2364, in run_wrapper
Job failed, going on with independent jobs.
Exiting because a job execution failed. Look above for error message
`
Could anyone please help me with this problem? Thank you very much!