pyranges / ncls

The Nested Containment List for Python. Basically a static interval-tree that is silly fast for both construction and lookups.
BSD 3-Clause "New" or "Revised" License
213 stars 20 forks source link

issue when installing from yaml file #50

Open vsbuffalo opened 1 year ago

vsbuffalo commented 1 year ago

Hey ncls folks,

I really like ncls but am having a very odd issue when installing it from an environment YAML file, which I need to use to include it as a dependency for a software package. I have created an MRE. Here is the test.yml file:

name: test
dependencies:
  - pip:
    - ncls

Note that it does work when I use:

$ pip install ncls
Collecting ncls
  Using cached ncls-0.0.66-cp310-cp310-macosx_11_0_arm64.whl
Requirement already satisfied: numpy in /Users/vsb/miniconda3/lib/python3.10/site-packages (from ncls) (1.24.2)
Installing collected packages: ncls
Successfully installed ncls-0.0.66

but when I create a new conda environment using this test.yml and mamba (I tried with conda too, but same problem), it compiles and hits an error:

$ mamba env create -n test -f test.yml
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
conda-forge/osx-arm64                                       Using cache
conda-forge/noarch                                          Using cache
bioconda/osx-arm64                                          Using cache
bioconda/noarch                                             Using cache
pkgs/r/osx-arm64                                              No change
pkgs/r/noarch                                                 No change
pkgs/main/noarch                                              No change
pkgs/main/osx-arm64                                           No change

Looking for: ['pip']

Transaction

  Prefix: /Users/vsb/miniconda3/envs/test

  Updating specs:

   - pip

  Package             Version  Build               Channel                     Size
─────────────────────────────────────────────────────────────────────────────────────
  Install:
─────────────────────────────────────────────────────────────────────────────────────

  + bzip2               1.0.8  h3422bc3_4          conda-forge/osx-arm64     Cached
  + ca-certificates  2023.5.7  hf0a4a13_0          conda-forge/osx-arm64     Cached
  + libexpat            2.5.0  hb7217d7_1          conda-forge/osx-arm64     Cached
  + libffi              3.4.2  h3422bc3_5          conda-forge/osx-arm64     Cached
  + libsqlite          3.41.2  hb31c410_1          conda-forge/osx-arm64     Cached
  + libzlib            1.2.13  h03a7124_4          conda-forge/osx-arm64     Cached
  + ncurses               6.3  h07bb92c_1          conda-forge/osx-arm64     Cached
  + openssl             3.1.0  h53f4e23_3          conda-forge/osx-arm64     Cached
  + pip                23.1.2  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + python             3.11.3  h1456518_0_cpython  conda-forge/osx-arm64     Cached
  + readline              8.2  h92ec313_1          conda-forge/osx-arm64     Cached
  + setuptools         67.7.2  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + tk                 8.6.12  he1e0b03_0          conda-forge/osx-arm64     Cached
  + tzdata              2023c  h71feb2d_0          conda-forge/noarch        Cached
  + wheel              0.40.0  pyhd8ed1ab_0        conda-forge/noarch        Cached
  + xz                  5.2.6  h57fd34a_0          conda-forge/osx-arm64     Cached

  Summary:

  Install: 16 packages

  Total download: 0 B

─────────────────────────────────────────────────────────────────────────────────────

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/Users/vsb/miniconda3/envs/test/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/vsb/src/cod_temp_cov/condaenv.y8g_6009.requirements.txt', '--exists-action=b']
Pip subprocess output:
Collecting ncls (from -r /Users/vsb/src/cod_temp_cov/condaenv.y8g_6009.requirements.txt (line 1))
  Using cached ncls-0.0.66.tar.gz (494 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting numpy (from ncls->-r /Users/vsb/src/cod_temp_cov/condaenv.y8g_6009.requirements.txt (line 1))
  Using cached numpy-1.24.3-cp311-cp311-macosx_11_0_arm64.whl (13.8 MB)
Building wheels for collected packages: ncls
  Building wheel for ncls (setup.py): started
  Building wheel for ncls (setup.py): finished with status 'error'
  Running setup.py clean for ncls
Failed to build ncls

Pip subprocess error:
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [87 lines of output]

      Warning: Cannot compile with Cython. Using legacy build.

      ncls/src/ncls.c:28981:43: warning: 'ob_shash' is deprecated [-Wdeprecated-declarations]
                  hash1 = ((PyBytesObject*)s1)->ob_shash;
                                                ^
      /Users/vsb/miniconda3/envs/test/include/python3.11/cpython/bytesobject.h:7:5: note: 'ob_shash' has been explicitly marked deprecated here
          Py_DEPRECATED(3.11) Py_hash_t ob_shash;
          ^
      /Users/vsb/miniconda3/envs/test/include/python3.11/pyport.h:336:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      ncls/src/ncls.c:28982:43: warning: 'ob_shash' is deprecated [-Wdeprecated-declarations]
                  hash2 = ((PyBytesObject*)s2)->ob_shash;
                                                ^
      /Users/vsb/miniconda3/envs/test/include/python3.11/cpython/bytesobject.h:7:5: note: 'ob_shash' has been explicitly marked deprecated here
          Py_DEPRECATED(3.11) Py_hash_t ob_shash;
          ^
      /Users/vsb/miniconda3/envs/test/include/python3.11/pyport.h:336:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      ncls/src/ncls.c:29355:23: error: no member named 'exc_type' in 'struct _err_stackitem'
          while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
                  ~~~~~~~~  ^
      ncls/src/ncls.c:29355:53: error: no member named 'exc_type' in 'struct _err_stackitem'
          while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
                                                ~~~~~~~~  ^
      ncls/src/ncls.c:29369:23: error: no member named 'exc_type' in 'struct _err_stackitem'
          *type = exc_info->exc_type;
                  ~~~~~~~~  ^
      ncls/src/ncls.c:29371:21: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          *tb = exc_info->exc_traceback;
                ~~~~~~~~  ^
      ncls/src/ncls.c:29385:26: error: no member named 'exc_type' in 'struct _err_stackitem'
          tmp_type = exc_info->exc_type;
                     ~~~~~~~~  ^
      ncls/src/ncls.c:29387:24: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          tmp_tb = exc_info->exc_traceback;
                   ~~~~~~~~  ^
      ncls/src/ncls.c:29388:15: error: no member named 'exc_type' in 'struct _err_stackitem'
          exc_info->exc_type = type;
          ~~~~~~~~  ^
      ncls/src/ncls.c:29390:15: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          exc_info->exc_traceback = tb;
          ~~~~~~~~  ^
      ncls/src/ncls.c:29447:30: error: no member named 'exc_type' in 'struct _err_stackitem'
              tmp_type = exc_info->exc_type;
                         ~~~~~~~~  ^
      ncls/src/ncls.c:29449:28: error: no member named 'exc_traceback' in 'struct _err_stackitem'
              tmp_tb = exc_info->exc_traceback;
                       ~~~~~~~~  ^
      ncls/src/ncls.c:29450:19: error: no member named 'exc_type' in 'struct _err_stackitem'
              exc_info->exc_type = local_type;
              ~~~~~~~~  ^
      ncls/src/ncls.c:29452:19: error: no member named 'exc_traceback' in 'struct _err_stackitem'
              exc_info->exc_traceback = local_tb;
              ~~~~~~~~  ^
      ncls/src/ncls.c:29485:26: error: no member named 'exc_type' in 'struct _err_stackitem'
          tmp_type = exc_info->exc_type;
                     ~~~~~~~~  ^
      ncls/src/ncls.c:29487:24: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          tmp_tb = exc_info->exc_traceback;
                   ~~~~~~~~  ^
      ncls/src/ncls.c:29488:15: error: no member named 'exc_type' in 'struct _err_stackitem'
          exc_info->exc_type = *type;
          ~~~~~~~~  ^
      ncls/src/ncls.c:29490:15: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          exc_info->exc_traceback = *tb;
          ~~~~~~~~  ^
      ncls/src/ncls.c:30888:5: error: incomplete definition of type 'struct _frame'
          __Pyx_PyFrame_SetLineNumber(py_frame, py_line);
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      ncls/src/ncls.c:432:62: note: expanded from macro '__Pyx_PyFrame_SetLineNumber'
        #define __Pyx_PyFrame_SetLineNumber(frame, lineno)  (frame)->f_lineno = (lineno)
                                                            ~~~~~~~^
      /Users/vsb/miniconda3/envs/test/include/python3.11/pytypedefs.h:22:16: note: forward declaration of 'struct _frame'
      typedef struct _frame PyFrameObject;
                     ^
      ncls/src/ncls.c:32961:20: error: no member named 'exc_type' in 'struct _err_stackitem'
          t = exc_state->exc_type;
              ~~~~~~~~~  ^
      ncls/src/ncls.c:32963:21: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          tb = exc_state->exc_traceback;
               ~~~~~~~~~  ^
      fatal error: too many errors emitted, stopping now [-ferror-limit=]
      2 warnings and 20 errors generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ncls
ERROR: Could not build wheels for ncls, which is required to install pyproject.toml-based projects

failed

CondaEnvException: Pip failed
endrebak commented 1 year ago

Thanks for reporting. I’ll have a look in less than a week.

Endre

On Wed, 10 May 2023 at 19:36, Vince Buffalo @.***> wrote:

Hey ncls folks,

I really like ncls but am having a very odd issue when installing it from an environment YAML file, which I need to use to include it as a dependency for a software package. I have created an MRE. Here is the test.yml file:

name: testdependencies:

  • pip:
    • ncls

Note that it does work when I use:

$ pip install ncls Collecting ncls Using cached ncls-0.0.66-cp310-cp310-macosx_11_0_arm64.whl Requirement already satisfied: numpy in /Users/vsb/miniconda3/lib/python3.10/site-packages (from ncls) (1.24.2) Installing collected packages: ncls Successfully installed ncls-0.0.66

but when I create a new conda environment using this test.yml and mamba (I tried with conda too, but same problem), it compiles and hits an error:

$ mamba env create -n test -f test.yml Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you. conda-forge/osx-arm64 Using cache conda-forge/noarch Using cache bioconda/osx-arm64 Using cache bioconda/noarch Using cache pkgs/r/osx-arm64 No change pkgs/r/noarch No change pkgs/main/noarch No change pkgs/main/osx-arm64 No change

Looking for: ['pip']

Transaction

Prefix: /Users/vsb/miniconda3/envs/test

Updating specs:

  • pip

    Package Version Build Channel Size ───────────────────────────────────────────────────────────────────────────────────── Install: ─────────────────────────────────────────────────────────────────────────────────────

    • bzip2 1.0.8 h3422bc3_4 conda-forge/osx-arm64 Cached
    • ca-certificates 2023.5.7 hf0a4a13_0 conda-forge/osx-arm64 Cached
    • libexpat 2.5.0 hb7217d7_1 conda-forge/osx-arm64 Cached
    • libffi 3.4.2 h3422bc3_5 conda-forge/osx-arm64 Cached
    • libsqlite 3.41.2 hb31c410_1 conda-forge/osx-arm64 Cached
    • libzlib 1.2.13 h03a7124_4 conda-forge/osx-arm64 Cached
    • ncurses 6.3 h07bb92c_1 conda-forge/osx-arm64 Cached
    • openssl 3.1.0 h53f4e23_3 conda-forge/osx-arm64 Cached
    • pip 23.1.2 pyhd8ed1ab_0 conda-forge/noarch Cached
    • python 3.11.3 h1456518_0_cpython conda-forge/osx-arm64 Cached
    • readline 8.2 h92ec313_1 conda-forge/osx-arm64 Cached
    • setuptools 67.7.2 pyhd8ed1ab_0 conda-forge/noarch Cached
    • tk 8.6.12 he1e0b03_0 conda-forge/osx-arm64 Cached
    • tzdata 2023c h71feb2d_0 conda-forge/noarch Cached
    • wheel 0.40.0 pyhd8ed1ab_0 conda-forge/noarch Cached
    • xz 5.2.6 h57fd34a_0 conda-forge/osx-arm64 Cached

    Summary:

    Install: 16 packages

    Total download: 0 B

─────────────────────────────────────────────────────────────────────────────────────

Downloading and Extracting Packages

Preparing transaction: done Verifying transaction: done Executing transaction: done Installing pip dependencies: - Ran pip subprocess with arguments: ['/Users/vsb/miniconda3/envs/test/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/vsb/src/cod_temp_cov/condaenv.y8g_6009.requirements.txt', '--exists-action=b'] Pip subprocess output: Collecting ncls (from -r /Users/vsb/src/cod_temp_cov/condaenv.y8g_6009.requirements.txt (line 1)) Using cached ncls-0.0.66.tar.gz (494 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting numpy (from ncls->-r /Users/vsb/src/cod_temp_cov/condaenv.y8g_6009.requirements.txt (line 1)) Using cached numpy-1.24.3-cp311-cp311-macosx_11_0_arm64.whl (13.8 MB) Building wheels for collected packages: ncls Building wheel for ncls (setup.py): started Building wheel for ncls (setup.py): finished with status 'error' Running setup.py clean for ncls Failed to build ncls

Pip subprocess error: error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [87 lines of output]

  Warning: Cannot compile with Cython. Using legacy build.

  ncls/src/ncls.c:28981:43: warning: 'ob_shash' is deprecated [-Wdeprecated-declarations]
              hash1 = ((PyBytesObject*)s1)->ob_shash;
                                            ^
  /Users/vsb/miniconda3/envs/test/include/python3.11/cpython/bytesobject.h:7:5: note: 'ob_shash' has been explicitly marked deprecated here
      Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      ^
  /Users/vsb/miniconda3/envs/test/include/python3.11/pyport.h:336:54: note: expanded from macro 'Py_DEPRECATED'
  #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                       ^
  ncls/src/ncls.c:28982:43: warning: 'ob_shash' is deprecated [-Wdeprecated-declarations]
              hash2 = ((PyBytesObject*)s2)->ob_shash;
                                            ^
  /Users/vsb/miniconda3/envs/test/include/python3.11/cpython/bytesobject.h:7:5: note: 'ob_shash' has been explicitly marked deprecated here
      Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      ^
  /Users/vsb/miniconda3/envs/test/include/python3.11/pyport.h:336:54: note: expanded from macro 'Py_DEPRECATED'
  #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                       ^
  ncls/src/ncls.c:29355:23: error: no member named 'exc_type' in 'struct _err_stackitem'
      while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
              ~~~~~~~~  ^
  ncls/src/ncls.c:29355:53: error: no member named 'exc_type' in 'struct _err_stackitem'
      while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
                                            ~~~~~~~~  ^
  ncls/src/ncls.c:29369:23: error: no member named 'exc_type' in 'struct _err_stackitem'
      *type = exc_info->exc_type;
              ~~~~~~~~  ^
  ncls/src/ncls.c:29371:21: error: no member named 'exc_traceback' in 'struct _err_stackitem'
      *tb = exc_info->exc_traceback;
            ~~~~~~~~  ^
  ncls/src/ncls.c:29385:26: error: no member named 'exc_type' in 'struct _err_stackitem'
      tmp_type = exc_info->exc_type;
                 ~~~~~~~~  ^
  ncls/src/ncls.c:29387:24: error: no member named 'exc_traceback' in 'struct _err_stackitem'
      tmp_tb = exc_info->exc_traceback;
               ~~~~~~~~  ^
  ncls/src/ncls.c:29388:15: error: no member named 'exc_type' in 'struct _err_stackitem'
      exc_info->exc_type = type;
      ~~~~~~~~  ^
  ncls/src/ncls.c:29390:15: error: no member named 'exc_traceback' in 'struct _err_stackitem'
      exc_info->exc_traceback = tb;
      ~~~~~~~~  ^
  ncls/src/ncls.c:29447:30: error: no member named 'exc_type' in 'struct _err_stackitem'
          tmp_type = exc_info->exc_type;
                     ~~~~~~~~  ^
  ncls/src/ncls.c:29449:28: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          tmp_tb = exc_info->exc_traceback;
                   ~~~~~~~~  ^
  ncls/src/ncls.c:29450:19: error: no member named 'exc_type' in 'struct _err_stackitem'
          exc_info->exc_type = local_type;
          ~~~~~~~~  ^
  ncls/src/ncls.c:29452:19: error: no member named 'exc_traceback' in 'struct _err_stackitem'
          exc_info->exc_traceback = local_tb;
          ~~~~~~~~  ^
  ncls/src/ncls.c:29485:26: error: no member named 'exc_type' in 'struct _err_stackitem'
      tmp_type = exc_info->exc_type;
                 ~~~~~~~~  ^
  ncls/src/ncls.c:29487:24: error: no member named 'exc_traceback' in 'struct _err_stackitem'
      tmp_tb = exc_info->exc_traceback;
               ~~~~~~~~  ^
  ncls/src/ncls.c:29488:15: error: no member named 'exc_type' in 'struct _err_stackitem'
      exc_info->exc_type = *type;
      ~~~~~~~~  ^
  ncls/src/ncls.c:29490:15: error: no member named 'exc_traceback' in 'struct _err_stackitem'
      exc_info->exc_traceback = *tb;
      ~~~~~~~~  ^
  ncls/src/ncls.c:30888:5: error: incomplete definition of type 'struct _frame'
      __Pyx_PyFrame_SetLineNumber(py_frame, py_line);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ncls/src/ncls.c:432:62: note: expanded from macro '__Pyx_PyFrame_SetLineNumber'
    #define __Pyx_PyFrame_SetLineNumber(frame, lineno)  (frame)->f_lineno = (lineno)
                                                        ~~~~~~~^
  /Users/vsb/miniconda3/envs/test/include/python3.11/pytypedefs.h:22:16: note: forward declaration of 'struct _frame'
  typedef struct _frame PyFrameObject;
                 ^
  ncls/src/ncls.c:32961:20: error: no member named 'exc_type' in 'struct _err_stackitem'
      t = exc_state->exc_type;
          ~~~~~~~~~  ^
  ncls/src/ncls.c:32963:21: error: no member named 'exc_traceback' in 'struct _err_stackitem'
      tb = exc_state->exc_traceback;
           ~~~~~~~~~  ^
  fatal error: too many errors emitted, stopping now [-ferror-limit=]
  2 warnings and 20 errors generated.
  error: command '/usr/bin/clang' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for ncls ERROR: Could not build wheels for ncls, which is required to install pyproject.toml-based projects

failed

CondaEnvException: Pip failed

— Reply to this email directly, view it on GitHub https://github.com/biocore-ntnu/ncls/issues/50, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHURUTDYBREWGECPI7ALIDXFPGZLANCNFSM6AAAAAAX5BECVI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

endrebak commented 1 year ago

Seems like the specific errors you are getting is due to https://bugs.python.org/issue45711

endrebak commented 1 year ago

(I deleted some red herring replies in this thread).

I have no solution now, but I think this will be fixed when we are able to build wheels.

endrebak commented 1 year ago

@vsbuffalo After we started building wheels, it works on my end.

Does it work for you too? If not, what wheels are you missing?