search

JuliaPy / PythonCall.jl

Python and Julia in harmony.
https://juliapy.github.io/PythonCall.jl/stable/
MIT License
722 stars 61 forks source link

`Using PythonCall` results in a NumPy compilation error #340

Open zornsllama opened 11 months ago

zornsllama commented 11 months ago

Affects: PythonCall

Describe the bug On doing using PythonCall, CondaPkg begins to set up a local Python environment. However, on reaching the installation of numpy, I receive the following compilation error:

Collecting numpy<1.19.0,>=1.15.4 (from ipfx)
  Using cached numpy-1.18.5.zip (5.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [60 lines of output]
      Running from numpy source directory.
      <string>:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      /tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py:75: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        required_version = LooseVersion('0.29.14')
      /tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py:77: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(cython_version) < required_version:
      warning: /tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/numpy/__init__.pxd:17:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: /tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/numpy/__init__.pxd:17:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          def __init__(self, seed=None):
              BitGenerator.__init__(self, seed)
              self.rng_state.pcg_state = &self.pcg64_random_state

              self._bitgen.state = <void *>&self.rng_state
              self._bitgen.next_uint64 = &pcg64_uint64
                                         ^
      ------------------------------------------------------------

      _pcg64.pyx:113:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'
      Processing numpy/random/_bounded_integers.pxd.in
      Processing numpy/random/mtrand.pyx
      Processing numpy/random/_pcg64.pyx
      Traceback (most recent call last):
        File "/tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py", line 238, in <module>
          main()
        File "/tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py", line 234, in main
          find_process_files(root_dir)
        File "/tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py", line 225, in find_process_files
          process(root_dir, fromfile, tofile, function, hash_db)
        File "/tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py", line 191, in process
          processor_function(fromfile, tofile)
        File "/tmp/pip-install-cgyfq94t/numpy_63725e2b74454648b28b02acfdf4db11/tools/cythonize.py", line 80, in process_pyx
          subprocess.check_call(
        File "/home/*/.julia/environments/v1.9/.CondaPkg/env/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/home/*/.julia/environments/v1.9/.CondaPkg/env/bin/python3.11', '-m', 'cython', '-3', '--fast-fail', '-o', '_pcg64.c', '_pcg64.pyx']' returned non-zero exit status 1.
      Cythonizing sources
      Traceback (most recent call last):
        File "/home/*/.julia/environments/v1.9/.CondaPkg/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/*/.julia/environments/v1.9/.CondaPkg/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/*/.julia/environments/v1.9/.CondaPkg/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-ngmf88ne/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 380, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-ngmf88ne/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 488, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-ngmf88ne/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 488, in <module>
        File "<string>", line 469, in setup_package
        File "<string>", line 275, in generate_cython
      RuntimeError: Running cythonize failed!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
ERROR: InitError: failed process: Process(`/home/*/.julia/environments/v1.9/.CondaPkg/env/bin/pip install ipfx pyabf`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:565 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:480
  [3] run(::Cmd)
    @ Base ./process.jl:477
  [4] _run(io::IO, cmd::Cmd, args::Any; flags::Any)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:370
  [5] (::CondaPkg.var"#29#30"{Base.TTY, Vector{String}, Vector{String}, Vector{String}})()
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:304
  [6] withenv(f::Function)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/env.jl:44
  [7] _resolve_pip_install(io::Any, pip_specs::Any, load_path::Any)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:301
  [8] resolve(; force::Bool, io::IO, interactive::Bool, dry_run::Bool)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:528
  [9] resolve()
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:384
 [10] envdir()
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/env.jl:70
 [11] init_context()
    @ PythonCall.C ~/.julia/packages/PythonCall/1f5yE/src/cpython/context.jl:63
 [12] __init__()
    @ PythonCall.C ~/.julia/packages/PythonCall/1f5yE/src/cpython/CPython.jl:21
 [13] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1074
 [14] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1020
 [15] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1471
 [16] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1748
 [17] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1625
 [18] macro expansion
    @ ./loading.jl:1613 [inlined]
 [19] macro expansion
    @ ./lock.jl:267 [inlined]
 [20] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1576
during initialization of module C

Your system Please provide detailed information about your system:

cjdoris commented 11 months ago

What happens if you try to create an equivalent environment by hand, like this?

conda create -n test sympy
conda activate test
pip install ipfx pyabf

If that fails, then the issue lies with something else, not PythonCall.

zornsllama commented 11 months ago

Do you mean to run this through CondaPkg.jl or an external Conda installation? If the former, "conda create" doesn't seem to be a recognized command. If the latter, I don't actually have an external Conda installation outside Julia.

cjdoris commented 11 months ago

Yeah the latter. Can you install conda (or mamba or micromamba) and try?

github-actions[bot] commented 10 months ago

This issue has been marked as stale because it has been open for 30 days with no activity. If the issue is still relevant then please leave a comment, or else it will be closed in 7 days.

github-actions[bot] commented 10 months ago

This issue has been closed because it has been stale for 7 days. You can re-open it if it is still relevant.