Error using pip install

[mariharris]

Hello, I've been trying to use pip install diploshic but I keep getting the following error:

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for diploshic Failed to build diploshic ERROR: Could not build wheels for diploshic, which is required to install pyproject.toml-based projects

I have also tried to clone and then install but I get the same error. I tried it on a Mac and on a linux-based cluster.

Any help would be appreciated. Thanks!

[andrewkern]

very strange. can you give me some more information here-- what version of python is this? what version of pip?

[mariharris]

I'm using Python 3.9.16 and pip 23.0.1

[andrewkern]

just tried to set this up on my macintosh laptop and it worked without issue using

 $ mamba create -n foo python=3.9.16
 $ mamba activate foo
 $ pip install diploshic

can you show me what you are doing in setting up your environment?

[mariharris]

I'm doing

$ conda create -n diploSHIC python=3.9.16
$ conda activate diploSHIC
$ pip install diploshic

[andrewkern]

i wonder if this is a conda issue?

Can you try it with mambaforge https://mamba.readthedocs.io/en/latest/installation.html

[mariharris]

Still same error. These are the errors that I get:

`Building wheels for collected packages: diploshic
  Building wheel for diploshic (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for diploshic (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [361 lines of output]
      <string>:2: DeprecationWarning:

        `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
        of the deprecation of `distutils` itself. It will be removed for
        Python >= 3.12. For older Python versions it will remain present.
        It is recommended to use `setuptools < 60.0` for those Python versions.
        For more details, see:
          https://numpy.org/devdocs/reference/distutils_status_migration.html
`

INFO: compile options: '-I/Users/marianaharris/mambaforge/envs/diploSHIC/include/python3.9 -c'
      extra options: '-mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq'
      WARN: CCompilerOpt.dist_test[630] : CCompilerOpt._dist_test_spawn[764] : Command (clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/marianaharris/mambaforge/envs/diploSHIC/include -arch x86_64 -I/Users/marianaharris/mambaforge/envs/diploSHIC/include -fPIC -O2 -isystem /Users/marianaharris/mambaforge/envs/diploSHIC/include -arch x86_64 -ftrapping-math -I/Users/marianaharris/mambaforge/envs/diploSHIC/include/python3.9 -c /private/var/folders/bq/v7pf8vvj4dn7081f07fwrmsm0000gn/T/pip-build-env-sh9yp10v/overlay/lib/python3.9/site-packages/numpy/distutils/checks/test_flags.c -o /var/folders/bq/v7pf8vvj4dn7081f07fwrmsm0000gn/T/tmpxr1ej7ay/private/var/folders/bq/v7pf8vvj4dn7081f07fwrmsm0000gn/T/pip-build-env-sh9yp10v/overlay/lib/python3.9/site-packages/numpy/distutils/checks/test_flags.o -MMD -MF /var/folders/bq/v7pf8vvj4dn7081f07fwrmsm0000gn/T/tmpxr1ej7ay/private/var/folders/bq/v7pf8vvj4dn7081f07fwrmsm0000gn/T/pip-build-env-sh9yp10v/overlay/lib/python3.9/site-packages/numpy/distutils/checks/test_flags.o.d -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq) failed with exit status 1 output ->
      clang: error: unknown argument: '-mavx5124fmaps'
      clang: error: unknown argument: '-mavx5124vnniw'

      WARN: CCompilerOpt.cc_test_flags[1081] : testing failed

      INFO: compile options: '-I/Users/marianaharris/mambaforge/envs/diploSHIC/include/python3.9 -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mno-mmx -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror=switch -Werror'
      INFO: get_default_fcompiler: matching types: '['gnu95', 'nag', 'nagfor', 'absoft', 'ibm', 'intel', 'gnu', 'g95', 'pg']'
      INFO: customize Gnu95FCompiler
      WARN: Could not locate executable gfortran
      WARN: Could not locate executable f95
      INFO: customize NAGFCompiler
      INFO: customize NAGFORCompiler
      WARN: Could not locate executable nagfor
      INFO: customize AbsoftFCompiler
      WARN: Could not locate executable f90
      WARN: Could not locate executable f77
      INFO: customize IBMFCompiler
      WARN: Could not locate executable xlf90
      WARN: Could not locate executable xlf
      INFO: customize IntelFCompiler
      WARN: Could not locate executable ifort
      WARN: Could not locate executable ifc
      INFO: customize GnuFCompiler
      WARN: Could not locate executable g77
      INFO: customize G95FCompiler
      WARN: Could not locate executable g95
      INFO: customize PGroupFCompiler
      WARN: Could not locate executable pgfortran
      WARN: don't know how to compile Fortran code on platform 'posix'
      warning: build_ext: f77_compiler=None is not available.

      INFO: building 'diploshic.shicstats' extension
      error: extension 'diploshic.shicstats' has Fortran sources but no Fortran compiler found
      INFO:
      ########### EXT COMPILER OPTIMIZATION ###########
      INFO: Platform      :
        Architecture: x64
        Compiler    : clang

      CPU baseline  :
        Requested   : 'min'
        Enabled     : SSE SSE2 SSE3
        Flags       : -msse -msse2 -msse3
        Extra checks: none

      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
        Generated   : none
      INFO: CCompilerOpt.cache_flush[857] : write cache to path -> /private/var/folders/bq/v7pf8vvj4dn7081f07fwrmsm0000gn/T/pip-install-6xt_tysl/diploshic_1db1df16ccf14958875a6a5d98d1a9cb/build/temp.macosx-10.9-x86_64-cpython-39/ccompiler_opt_cache_ext.py
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for diploshic
Failed to build diploshic
ERROR: Could not build wheels for diploshic, which is required to install pyproject.toml-based projects

[andrewkern]

this is strange because pip seems to be trying to build from source, rather than just download the binary as it should do.

to diagnose this-- 1) are you trying to install from the diploshic source dir? I don't think this is what you'd want to do 2) can you clear your pip cache of any diploshic related stuff and then retry? i think the command would be pip cache remove diploshic 3) alternatively you can disable building from source I think with pip --no-cache-dir

[mariharris]

ok yeah, for some reason pip was trying to build from source. I tried pip install --no-build-isolation diploshic And I think that did it.

Thanks for your help!

[MattHartfield]

I'm trying to install diploS/HIC and I'm also having trouble doing so, in a similar vein to the above problem. I tried cloning the repo into a new directory:

conda create -n diploshic python=3.9 --yes

git clone https://github.com/kern-lab/diploSHIC.git
cd diploSHIC 
pip install .

but after doing so I get the same error message about numpy.distutils being deprecated. Furthermore, when I tried to use pip install --no-build-isolation diploshic I get the following error:

Collecting diploshic Using cached diploSHIC-1.0.4.tar.gz (13.9 MB) Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [13 lines of output] Traceback (most recent call last): File "/ceph/users/mhartfield/.conda/envs/diploshic/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/ceph/users/mhartfield/.conda/envs/diploshic/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/ceph/users/mhartfield/.conda/envs/diploshic/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) File "/ceph/users/mhartfield/.conda/envs/diploshic/lib/python3.9/site-packages/setuptools/build_meta.py", line 380, in prepare_metadata_for_build_wheel self.run_setup() File "/ceph/users/mhartfield/.conda/envs/diploshic/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup exec(code, locals()) File "", line 2, in ModuleNotFoundError: No module named 'numpy' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

I also tried pip install --no-cache-dir . but get the same problem as in previous posts. I initially tried this on a linux-based cluster but get the same errors when trying to install on my Mac (OSX 13.4 Ventura).

Apologies in advance if it's considered 'bad git etiquette' to reopen this report, but given the similarity of the issues I thought it best to post a new message here.

[andrewkern]

now worries! again it looks like a pip caching issue as it is saying:

Using cached diploSHIC-1.0.4.tar.gz

can you delete the pip cache and report back? i'm guessing it will be the one in your home directory on a mac or linux system at ~/.cache/

[MattHartfield]

Hi, yes this seems to work! I was able to install it after deleting the pip cache.

I had to do one other thing to get it to work - I'm principally running diploSHIC on a cluster where one submits jobs to a queue. If I ran it direct from the command line then it worked, but when I tried running it via a shell script that was submitted to a queue then it would report an error saying that 'libgfortran3' was not found. I eventually got it to work by installing libgfortran via conda. This could easily be a quirk of how diploSHIC runs on our cluster, but I thought it's worth reporting in any case (when googling for a solution I came across posts about how this library needed updating on newer versions of Ubuntu - maybe that's relevant here?).