search

mariomulansky / PySpike

Python implementation of spike distance metrics
http://mariomulansky.github.io/PySpike
Other
71 stars 30 forks source link

Cython 3.x: build fails #70

Closed penguinpee closed 4 months ago

penguinpee commented 1 year ago

Fedora has updated to Cython 3.x in rawhide (development branch) and f39. It seems PySpike is not ready yet for Cython 3.x:

  running build_ext
  Traceback (most recent call last):
    File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 416, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
      self.run_setup()
    File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 488, in run_setup
      self).run_setup(setup_script=setup_script)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 72, in <module>
    File "/usr/lib/python3.12/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.12/site-packages/setuptools/dist.py", line 1244, in run_command
      super().run_command(command)
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.12/site-packages/wheel/bdist_wheel.py", line 343, in run
      self.run_command("build")
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.12/site-packages/setuptools/dist.py", line 1244, in run_command
      super().run_command(command)
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.12/site-packages/setuptools/dist.py", line 1244, in run_command
      super().run_command(command)
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib64/python3.12/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
      new_ext = cythonize(
                ^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 990, in cythonize
      module_list, module_metadata = create_extension_list(
                                     ^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 839, in create_extension_list
      kwds = deps.distutils_info(file, aliases, base).values
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 691, in distutils_info
      return (self.transitive_merge(filename, self.distutils_info0, DistutilsInfo.merge)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 700, in transitive_merge
      return self.transitive_merge_helper(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 712, in transitive_merge_helper
      for next in outgoing(node):
                  ^^^^^^^^^^^^^^
    File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 613, in cimported_files
      pxd_file = self.find_pxd(module, filename)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
    File "/usr/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 594, in find_pxd
      pxd = self.context.find_pxd_file(relative, source_file_path=filename)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Compiler/Main.py", line 281, in find_pxd_file
      pxd = self.search_include_directories(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.12/site-packages/Cython/Compiler/Main.py", line 309, in search_include_directories
      return search_include_directories(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "Cython/Utils.py", line 90, in Cython.Utils.cached_function.wrapper
    File "/usr/lib64/python3.12/site-packages/Cython/Compiler/Main.py", line 702, in search_include_directories
      path = os.path.join(dirname, dotted_filename)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen posixpath>", line 76, in join
  TypeError: expected str, bytes or os.PathLike object, not numpy_include
  error: subprocess-exited-with-error
musicinmybrain commented 6 months ago

This project is using a lazily-evaluated proxy for the numpy include path:

https://github.com/mariomulansky/PySpike/blob/master/setup.py#L24-L28

The class is instantiated in the include_dirs list, and then the numpy import and the call to numpy.get_include() are supposed to happen when the instance gets “stringified” as in str(foo) or '{}'.format(foo).

https://github.com/mariomulansky/PySpike/blob/master/setup.py#L90

Unfortunately, current versions of Cython don’t just blindly convert anything in the sequence of include directories to a string, but validate it against a list of supported types, thus:

TypeError: expected str, bytes or os.PathLike object, not numpy_include

A simple downstream patch would be to just replace numpy_include() with str(numpy_include()) to force immediate evaluation, since we can guarantee that numpy is already available in the environment.

To maintain the original behavior, though, we can adjust numpy_include to be an instance of os.PathLike. That should fix Cython 3 compatibility in a backwards-compatible way. I’ll open a PR.

musicinmybrain commented 6 months ago

Hmm: https://github.com/mariomulansky/PySpike/pull/68 is exactly what I was going to do, except that I was going to replace __str__() with __fspath__(). I opened my own version of the PR anyway; either is fine.