search

ythuang0522 / homopolish

High-quality Nanopore-only genome polisher
GNU General Public License v3.0
65 stars 12 forks source link

ImportError: cannot import name 'cached_property' from 'functools' #61

Open AnyaKovalenko opened 9 months ago

AnyaKovalenko commented 9 months ago

Dear Community,

please can you help with the following issue: <ImportError: cannot import name 'cached_property' from 'functools'>. Please see bellow:

$ homopolish polish -a barcodeXX.consensus.fasta -m R10.3.pkl -l reference_list_for_homopolish/virus.fasta -t 4 -o barcodeXX_homopolish 

Traceback (most recent call last):
  File "/home/annako/miniconda3/envs/homopolish/bin/homopolish", line 6, in <module>
    from homopolish.homopolish import main
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/homopolish/homopolish.py", line 4, in <module>
    from modules.polish_interface import polish_genome
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/modules/polish_interface.py", line 11, in <module>
    from modules import polish
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/modules/polish.py", line 5, in <module>
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/more_itertools/__init__.py", line 3, in <module>
    from .more import *  # noqa
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/more_itertools/more.py", line 5, in <module>
    from functools import cached_property, partial, reduce, wraps
ImportError: cannot import name 'cached_property' from 'functools' (/home/annako/miniconda3/envs/homopolish/lib/python3.7/functools.py)

Has anyone faced with the same issue? please help me to find out how to resolve this. Many thanks and very looking forward to receiving the feedback! Best wishes!

ythuang0522 commented 9 months ago

@AnyaKovalenko Can you upgrade your python to 3.8 or after? The cached_property seems introduced at 3.8. If still unresolved, can you provide the draft genome for us to debug?

AnyaKovalenko commented 9 months ago

Hi @ythuang0522 thank you. I updated python to 3.8 and this helped. But now another error <'ModuleNotFoundError: No module named 'sklearn.svm.classes'>, please see below. 


[2023/11/22 05:33] INFO: RUN-ID: NC_039477.1
NC_039477.1
/mnt/d/university/Metagenomics/analysis/BC01_homopolish/debug

[M::mm_idx_gen::0.017*0.21] collected minimizers
[M::mm_idx_gen::0.019*0.30] sorted minimizers
[M::main::0.019*0.31] loaded/built the index for 1 target sequence(s)
[M::mm_mapopt_update::0.019*0.31] mid_occ = 50
[M::mm_idx_stat] kmer size: 19; skip: 19; is_hpc: 0; #seq: 1
[M::mm_idx_stat::0.019*0.32] distinct minimizers: 762 (100.00% are singletons); average occurrences: 1.000; average spacing: 9.920; total length: 7559
[M::worker_pipeline::4.761*2.66] mapped 2890 sequences
[M::main] Version: 2.26-r1175
[M::main] CMD: minimap2 -cx asm5 --cs=long -t 4 /mnt/d/university/Metagenomics/analysis/BC01_homopolish/debug/NC_039477.1/NC_039477.1.fasta /mnt/d/university/Metagenomics/analysis/BC01_homopolish/debug/NC_039477.1/All_homologous_sequences.fna.gz
[M::main] Real time: 4.763 sec; CPU: 12.689 sec; Peak RSS: 0.424 GB
4.811844825744629
[2023/11/22 05:33] INFO: Stage: Homologous retrieval
TIME Homologous retrieval: 0 MINS 3 SECS.
[2023/11/22 05:34] INFO: Stage: Prediction
Traceback (most recent call last):
  File "/home/annako/miniconda3/envs/homopolish/bin/homopolish", line 10, in <module>
    sys.exit(main())
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/homopolish/homopolish.py", line 50, in main
    polish_genome(FLAGS.mash_screen, FLAGS.assembly, __pkg_path__, FLAGS.sketch_path, FLAGS.genus, FLAGS.threads, \
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 326, in polish_genome
    out = local_DB(out, assembly_name, output_dir_debug, mash_screen, assembly, model_path, sketch_path, genus_species, threads, output_dir, minimap_args, mash_threshold, download_contig_nums, debug, meta, local_DB_path)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 307, in local_DB
    out.append(check_homopolish(paf, contig_name, contig_output_dir, contig, minimap_args, threads, db_path, model_path))
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 131, in check_homopolish
    finish = homopolish(contig_name, minimap_args, threads, db_path, model_path, contig_output_dir, dataframe)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 91, in homopolish
    result = prediction.predict(dataframe, model_path, threads, contig_output_dir)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/prediction.py", line 19, in predict
    model = joblib.load(model)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 658, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 577, in _unpickle
    obj = unpickler.load()
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/pickle.py", line 1212, in load
    dispatch[key[0]](self)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/pickle.py", line 1528, in load_global
    klass = self.find_class(module, name)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/pickle.py", line 1579, in find_class
    __import__(module, level=0)
ModuleNotFoundError: No module named 'sklearn.svm.classes'

Please can you help how to deal with this. Many thanks! Very looking forward to hearing from you.

AnyaKovalenko commented 9 months ago

@ythuang0522 what is the best way to send you the draft genome? can you share your email and I will send through it? thank you

AnyaKovalenko commented 9 months ago

I think the scikit-learn versions used in model and program loading it are different. Isn't it? The scikit-learn was updated together with the python (the current scikit-learn version is 1.3.0)

ythuang0522 commented 9 months ago

Yes. You should use the conda virtual env as the scikit-learn model cannot be used in some versions. Did you use the recommended conda env conda env create -f environment.yml? If so, please send your genome to ythuang@ccu.edu.tw. Thanks.

AnyaKovalenko commented 9 months ago

Many thanks @ythuang0522 I sent the genomes. I use conda env for homopolish, but installed it through the mamba create -n homopolish -c conda-forge -c bioconda homopolish Can it be the issue?

Mossy-Frog commented 7 months ago

Hi ! I know I am late, but if anyone still has this problem, just like I had this morning, a possible solution is to : mamba remove python mamba install "python==3.8" Since it's a problem with the version of python, I uninstalled the previous python(s) and installed one that seemed to be compatible with every packages in homopolish

AnyaKovalenko commented 7 months ago

@Mossy-Frog thank you so much! It's never too late :)

AnyaKovalenko commented 6 months ago

Hi ! I know I am late, but if anyone still has this problem, just like I had this morning, a possible solution is to : mamba remove python mamba install "python==3.8" Since it's a problem with the version of python, I uninstalled the previous python(s) and installed one that seemed to be compatible with every packages in homopolish

Hi @Mossy-Frog I used these and still have an issue. So now the error: homopolish: command not found Did you have the same? thank you

AnyaKovalenko commented 6 months ago

okay, finally it was figured out! Just run : python3 homopolish.py polish -a yourgenome.fasta -m <select-model> -o youroutput instead of : homopolish polish -a yourgenome.fasta -m <select-model> -o youroutput

Mossy-Frog commented 6 months ago

okay, finally it was figured out! Just run : python3 homopolish.py polish -a yourgenome.fasta -m <select-model> -o youroutput instead of : homopolish polish -a yourgenome.fasta -m <select-model> -o youroutput

Huh, weird, I do not have this issue at all, I wonder what causes this ?

JonathanAbrahams1337 commented 5 months ago

62 Similar to my experience too.

Can the README be updated?

ythuang0522 commented 5 months ago

62 Similar to my experience too.

Can the README be updated?

We found several issues associated with the Python version as well as outdated packages in the conda-forge channel. e.g.,

  1. The conda-forge only includes libedit upto 3.1.2017, while some packages required >=3.1.20210910.
  2. The cached_property package requires python3.8 or later.

We have updated the installation instructions in README according to the issues we have found. The following instructs work at our env. conda create -n homopolish -c conda-forge -c bioconda -c defaults more-itertools=8.4.0 homopolish=0.4.1 or mamba create -n homopolish_mamba_auto_0_4_1 -c conda-forge -c bioconda -c defaults python==3.8.16 homopolish==0.4.1

Thanks for all the feedback.

JonathanAbrahams1337 commented 5 months ago

Great, thanks. Will give it a go.

Perhaps the inclusion of some test data can mean we can run the tool on the same data and test just the environment rather than the data+enviornment together. Will be very helpful for new users to ensure their install/conda env works.

JonathanAbrahams1337 commented 5 months ago

I am still having troubles with this, many parts of the files contain errors in my env, due to wrong package versions. Can you post your environment? (conda env export)


name: homopolish_mamba_auto_0_4_1_26.03.2024
channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_kmp_llvm
  - arrow-cpp=10.0.1=h3e2b116_4_cpu
  - aws-c-auth=0.6.21=hd93a3ba_3
  - aws-c-cal=0.5.20=hd3b2fe5_3
  - aws-c-common=0.8.5=h166bdaf_0
  - aws-c-compression=0.2.16=hf5f93bc_0
  - aws-c-event-stream=0.2.17=h57874a7_1
  - aws-c-http=0.7.0=h96ef541_0
  - aws-c-io=0.13.12=h57ca295_1
  - aws-c-mqtt=0.7.13=h0b5698f_12
  - aws-c-s3=0.2.1=h5d7e488_4
  - aws-c-sdkutils=0.1.7=hf5f93bc_0
  - aws-checksums=0.1.14=h6027aba_0
  - aws-crt-cpp=0.18.16=h26430d7_8
  - aws-sdk-cpp=1.9.379=h56135f1_7
  - biopython=1.83=py38h01eb140_0
  - blas=2.120=openblas
  - blas-devel=3.9.0=20_linux64_openblas
  - boost=1.70.0=py38h9de70de_1
  - boost-cpp=1.70.0=h8e57a91_2
  - brotli=1.0.9=h166bdaf_9
  - brotli-bin=1.0.9=h166bdaf_9
  - brotli-python=1.0.9=py38hfa26641_9
  - brotlipy=0.7.0=py38h01eb140_1006
  - bzip2=1.0.8=hd590300_5
  - c-ares=1.27.0=hd590300_0
  - ca-certificates=2024.2.2=hbcca054_0
  - capnproto=0.10.2=h6239696_0
  - certifi=2024.2.2=pyhd8ed1ab_0
  - cffi=1.16.0=py38h6d47a40_0
  - chardet=5.2.0=py38h578d9bd_1
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - cryptography=39.0.0=py38h1724139_0
  - curl=7.87.0=h6312ad2_0
  - double-conversion=3.3.0=h59595ed_0
  - fastani=1.34=h4dfc31f_1
  - feather-format=0.4.1=pyh9f0ad1d_0
  - gflags=2.2.2=he1b5a44_1004
  - glog=0.6.0=h6f12383_0
  - grpc-cpp=1.51.1=h3e2b116_0
  - gsl=2.7=he838d99_0
  - homopolish=0.4.1=pyhdfd78af_0
  - icu=64.2=he1b5a44_1
  - idna=3.6=pyhd8ed1ab_0
  - joblib=1.3.2=pyhd8ed1ab_0
  - k8=0.2.5=hdcf5f25_4
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.20.1=hf9c8cef_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - libabseil=20220623.0=cxx17_h05df665_6
  - libarrow=10.0.1=hee49ebd_4_cpu
  - libblas=3.9.0=20_linux64_openblas
  - libbrotlicommon=1.0.9=h166bdaf_9
  - libbrotlidec=1.0.9=h166bdaf_9
  - libbrotlienc=1.0.9=h166bdaf_9
  - libcblas=3.9.0=20_linux64_openblas
  - libcrc32c=1.1.2=h9c3ff4c_0
  - libcurl=7.87.0=h6312ad2_0
  - libdeflate=1.13=h166bdaf_0
  - libedit=3.1.20230828=h5eee18b_0
  - libev=4.33=hd590300_2
  - libevent=2.1.10=h9b69904_4
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=13.2.0=h807b86a_5
  - libgfortran-ng=13.2.0=h69a702a_5
  - libgfortran5=13.2.0=ha4646dd_5
  - libgoogle-cloud=2.5.0=h5d50b59_1
  - libgrpc=1.51.1=h05bd8bd_0
  - liblapack=3.9.0=20_linux64_openblas
  - liblapacke=3.9.0=20_linux64_openblas
  - libnghttp2=1.51.0=hdcd2b5c_0
  - libopenblas=0.3.25=pthreads_h413a1c8_0
  - libprotobuf=3.21.12=hfc55251_2
  - libsqlite=3.45.2=h2797004_0
  - libssh2=1.10.0=haa6b8db_3
  - libstdcxx-ng=13.2.0=h7e041cc_5
  - libthrift=0.16.0=h491838f_2
  - libutf8proc=2.8.0=h166bdaf_0
  - libzlib=1.2.13=hd590300_5
  - llvm-openmp=18.1.2=h4dfa4b3_0
  - lz4-c=1.9.4=hcb278e6_0
  - mash=2.3=hd3113c8_6
  - minimap2=2.27=he4a0461_1
  - more-itertools=10.2.0=pyhd8ed1ab_0
  - ncbi-datasets-cli=16.10.1=ha770c72_2
  - ncurses=6.4.20240210=h59595ed_0
  - numpy=1.24.3=py38hf838250_0
  - numpy-base=1.24.3=py38h1e6e340_0
  - openblas=0.3.25=pthreads_h7a3da1a_0
  - openssl=1.1.1w=hd590300_0
  - orc=1.8.2=hfdbbad2_2
  - packaging=24.0=pyhd8ed1ab_0
  - pandas=2.0.3=py38h01efb38_1
  - parquet-cpp=1.5.1=2
  - pip=24.0=pyhd8ed1ab_0
  - platformdirs=4.2.0=pyhd8ed1ab_0
  - pooch=1.8.1=pyhd8ed1ab_0
  - pyarrow=10.0.1=py38hf54eea0_4_cpu
  - pycparser=2.21=pyhd8ed1ab_0
  - pycurl=7.45.1=py38h61f0cdf_3
  - pyopenssl=23.2.0=pyhd8ed1ab_1
  - pysam=0.21.0=py38h1c8baaf_0
  - pysocks=1.7.1=pyha2e5f31_6
  - python=3.8.16=h7a1cb2a_3
  - python-dateutil=2.9.0=pyhd8ed1ab_0
  - python-tzdata=2024.1=pyhd8ed1ab_0
  - python-wget=3.2=py_0
  - python_abi=3.8=2_cp38
  - pytz=2024.1=pyhd8ed1ab_0
  - re2=2022.06.01=h27087fc_1
  - readline=8.2=h8228510_1
  - requests=2.31.0=pyhd8ed1ab_0
  - s2n=1.3.31=hae46d1a_0
  - scikit-learn=1.3.2=py38ha25d942_2
  - scipy=1.10.1=py38h59b608b_3
  - setuptools=69.2.0=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.1.10=h9fff704_0
  - sqlite=3.45.2=h2c6b66d_0
  - threadpoolctl=3.4.0=pyhc1e730c_0
  - thrift-compiler=0.16.0=h491838f_2
  - thrift-cpp=0.16.0=h3e2b116_2
  - tk=8.6.13=noxft_h4845f30_101
  - tqdm=4.66.2=pyhd8ed1ab_0
  - uriparser=0.9.7=h59595ed_1
  - urllib3=2.2.1=pyhd8ed1ab_0
  - wheel=0.43.0=pyhd8ed1ab_0
  - xz=5.2.10=h5eee18b_1
  - zlib=1.2.13=hd590300_5
  - zstd=1.5.5=hfc55251_0
prefix: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX