search

nanoporetech / medaka

Sequence correction provided by ONT Research
https://nanoporetech.com
Other
391 stars 73 forks source link

installing medaka (with pip) on M1 #512

Open stephrom opened 2 weeks ago

stephrom commented 2 weeks ago

Medaka is a Research Release.

Research releases are provided as technology demonstrators to provide early access to features or stimulate Community development of tools. Support for this software will be minimal and is only provided directly by the developers. Feature requests, improvements, and discussions are welcome and can be implemented by forking and pull requests. However much as we would like to rectify every issue and piece of feedback users may have, the developers may have limited resource for support of this software. Research releases may be unstable and subject to rapid iteration by Oxford Nanopore Technologies.

Please ensure that you are using the most recent version of medaka before filing a bug report. The most recent version can be found on the release page. If you are not using the most recent release, and file a issue regardless the most likely response from our developers will be to ask you to first upgrade.

Please ensure also to provide the information below, not doing so will likely result in a request for the information.

Describe the bug A clear and concise description of what the bug is including the command that you have run.

Logging Please attach any relevant logging messages. (Use ``` before and after code blocks).

Environment (if you do not have a GPU, write No GPU):

Additional context Add any other context about the problem here.

Dear,

I'm trying to install medaka (via pip install medaka) on a MacBook pro M1max (32Gb RAM) OS=sonoma 14.5. All dependancies are installed (most via homebrew). But during the configuration phase, the system doesn't seem to find libhts.a (although present in the homebrew/lib directory). I guess it then tries to install is from scratch (from samtools), but not in a way that works for an M1 cpu. Also the version from samtools I have installed in 1.20 (and htslib/1.20)

Is there a way to go around/fix the issue?

best regards

.... Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /opt/homebrew/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow-macos~=2.10.0->medaka) (0.5.0rc2) Requirement already satisfied: oauthlib>=3.0.0 in /opt/homebrew/lib/python3.10/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow-macos~=2.10.0->medaka) (3.2.2) Using cached cffi-1.15.0-cp310-cp310-macosx_11_0_arm64.whl (173 kB) Using cached h5py-3.10.0-cp310-cp310-macosx_11_0_arm64.whl (2.7 MB) Using cached tensorflow_macos-2.10.0-cp310-cp310-macosx_12_0_arm64.whl (211.5 MB) Using cached gast-0.4.0-py3-none-any.whl (9.8 kB) Using cached keras-2.10.0-py2.py3-none-any.whl (1.7 MB) Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB) Using cached protobuf-3.19.6-py2.py3-none-any.whl (162 kB) Using cached tensorboard-2.10.1-py3-none-any.whl (5.9 MB) Using cached tensorflow_estimator-2.10.0-py2.py3-none-any.whl (438 kB) Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB) Using cached tensorboard_data_server-0.6.1-py3-none-any.whl (2.4 kB) Building wheels for collected packages: medaka Building wheel for medaka (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for medaka (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [117 lines of output] /opt/homebrew/Cellar/python@3.10/3.10.14/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cffi/cparser.py:154: UserWarning: String literal found in cdef() or type source. String literals are ignored here, but you should remove them anyway because some character sequences confuse pre-parsing. warnings.warn("String literal found in cdef() or type source. " Bundling models: ['r103_fast_g507', 'r103_fast_snp_g507', 'r103_fast_variant_g507', 'r103_hac_g507', 'r103_hac_snp_g507', 'r103_hac_variant_g507', 'r103_min_high_g345', 'r103_min_high_g360', 'r103_prom_high_g360', 'r103_prom_snp_g3210', 'r103_prom_variant_g3210', 'r103_sup_g507', 'r103_sup_snp_g507', 'r103_sup_variant_g507', 'r1041_e82_260bps_fast_g632', 'r1041_e82_260bps_fast_variant_g632', 'r1041_e82_260bps_hac_g632', 'r1041_e82_260bps_hac_v4.0.0', 'r1041_e82_260bps_hac_v4.1.0', 'r1041_e82_260bps_hac_variant_g632', 'r1041_e82_260bps_hac_variant_v4.1.0', 'r1041_e82_260bps_joint_apk_ulk_v5.0.0', 'r1041_e82_260bps_sup_g632', 'r1041_e82_260bps_sup_v4.0.0', 'r1041_e82_260bps_sup_v4.1.0', 'r1041_e82_260bps_sup_variant_g632', 'r1041_e82_260bps_sup_variant_v4.1.0', 'r1041_e82_400bps_fast_g615', 'r1041_e82_400bps_fast_g632', 'r1041_e82_400bps_fast_variant_g615', 'r1041_e82_400bps_fast_variant_g632', 'r1041_e82_400bps_hac_g615', 'r1041_e82_400bps_hac_g632', 'r1041_e82_400bps_hac_v4.0.0', 'r1041_e82_400bps_hac_v4.1.0', 'r1041_e82_400bps_hac_v4.2.0', 'r1041_e82_400bps_hac_v4.3.0', 'r1041_e82_400bps_hac_v5.0.0', 'r1041_e82_400bps_hac_variant_g615', 'r1041_e82_400bps_hac_variant_g632', 'r1041_e82_400bps_hac_variant_v4.1.0', 'r1041_e82_400bps_hac_variant_v4.2.0', 'r1041_e82_400bps_hac_variant_v4.3.0', 'r1041_e82_400bps_hac_variant_v5.0.0', 'r1041_e82_400bps_sup_g615', 'r1041_e82_400bps_sup_v4.0.0', 'r1041_e82_400bps_sup_v4.1.0', 'r1041_e82_400bps_sup_v4.2.0', 'r1041_e82_400bps_sup_v4.3.0', 'r1041_e82_400bps_sup_v5.0.0', 'r1041_e82_400bps_sup_variant_g615', 'r1041_e82_400bps_sup_variant_v4.1.0', 'r1041_e82_400bps_sup_variant_v4.2.0', 'r1041_e82_400bps_sup_variant_v4.3.0', 'r1041_e82_400bps_sup_variant_v5.0.0', 'r104_e81_fast_g5015', 'r104_e81_fast_variant_g5015', 'r104_e81_hac_g5015', 'r104_e81_hac_variant_g5015', 'r104_e81_sup_g5015', 'r104_e81_sup_g610', 'r104_e81_sup_variant_g610', 'r10_min_high_g303', 'r10_min_high_g340', 'r941_e81_fast_g514', 'r941_e81_fast_variant_g514', 'r941_e81_hac_g514', 'r941_e81_hac_variant_g514', 'r941_e81_sup_g514', 'r941_e81_sup_variant_g514', 'r941_min_fast_g303', 'r941_min_fast_g507', 'r941_min_fast_snp_g507', 'r941_min_fast_variant_g507', 'r941_min_hac_g507', 'r941_min_hac_snp_g507', 'r941_min_hac_variant_g507', 'r941_min_high_g303', 'r941_min_high_g330', 'r941_min_high_g340_rle', 'r941_min_high_g344', 'r941_min_high_g351', 'r941_min_high_g360', 'r941_min_sup_g507', 'r941_min_sup_snp_g507', 'r941_min_sup_variant_g507', 'r941_prom_fast_g303', 'r941_prom_fast_g507', 'r941_prom_fast_snp_g507', 'r941_prom_fast_variant_g507', 'r941_prom_hac_g507', 'r941_prom_hac_snp_g507', 'r941_prom_hac_variant_g507', 'r941_prom_high_g303', 'r941_prom_high_g330', 'r941_prom_high_g344', 'r941_prom_high_g360', 'r941_prom_high_g4011', 'r941_prom_snp_g303', 'r941_prom_snp_g322', 'r941_prom_snp_g360', 'r941_prom_sup_g507', 'r941_prom_sup_snp_g507', 'r941_prom_sup_variant_g507', 'r941_prom_variant_g303', 'r941_prom_variant_g322', 'r941_prom_variant_g360', 'r941_sup_plant_g610', 'r941_sup_plant_variant_g610'] running bdist_wheel running build running build_py creating build creating build/lib.macosx-14-arm64-cpython-310 creating build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/tandem.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/options.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/vcf.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/labels.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/align.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/models.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/init.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/features.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/prediction.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/rle.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/medaka.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/common.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/wrappers.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/stitch.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/keras_ext.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/smolecule.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/variant.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/datastore.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/training.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/executor.py -> build/lib.macosx-14-arm64-cpython-310/medaka copying medaka/medaka_counts.py -> build/lib.macosx-14-arm64-cpython-310/medaka creating build/lib.macosx-14-arm64-cpython-310/medaka/data copying medaka/data/r1041_e82_400bps_hac_v5.0.0_model.tar.gz -> build/lib.macosx-14-arm64-cpython-310/medaka/data copying medaka/data/r1041_e82_400bps_hac_variant_v5.0.0_model.tar.gz -> build/lib.macosx-14-arm64-cpython-310/medaka/data copying medaka/data/r1041_e82_400bps_sup_v5.0.0_model.tar.gz -> build/lib.macosx-14-arm64-cpython-310/medaka/data copying medaka/data/r1041_e82_400bps_sup_variant_v5.0.0_model.tar.gz -> build/lib.macosx-14-arm64-cpython-310/medaka/data running build_ext generating cffi module 'build/temp.macosx-14-arm64-cpython-310/libmedaka.c' creating build/temp.macosx-14-arm64-cpython-310 Compiling htslib using Makefile cd submodules; \ curl -L -o samtools-1.14.tar.bz2 https://github.com/samtools/samtools/releases/download/1.14/samtools-1.14.tar.bz2; \ tar -xjf samtools-1.14.tar.bz2; \ rm samtools-1.14.tar.bz2 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed

    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

   51 7563k   51 3871k    0     0  3926k      0  0:00:01 --:--:--  0:00:01 3926k
  100 7563k  100 7563k    0     0  6768k      0  0:00:01  0:00:01 --:--:-- 27.5M
  # this is required only to add in -fpic so we can build python module
  Making libhts.a
  cd submodules/samtools-1.14/htslib-1.14/ \
              && CFLAGS="-fpic -O3 -std=c99 " ./configure  \
              && make -j 4
  checking for gcc... /opt/homebrew/bin/clang
  checking whether the C compiler works... no
  configure: error: in `/private/var/folders/8b/g7kmwclj051fq1xjkjtrtls00000gn/T/pip-install-5s7oaa1j/medaka_c632ed6f0c3a414d88cbc09d1d86ad22/submodules/samtools-1.14/htslib-1.14':
  configure: error: C compiler cannot create executables
  See `config.log' for more details
  make: *** [libhts.a] Error 77
  Traceback (most recent call last):
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel

// ...... // raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['make', 'libhts.a']' returned non-zero exit status 2. [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for medaka Failed to build medaka ERROR: Could not build wheels for medaka, which is required to install pyproject.toml-based projects

cjw85 commented 2 weeks ago

This thread may be helpful to you:

https://github.com/nanoporetech/medaka/issues/456#issuecomment-1711947247