ninia / jep

Embed Python in Java
Other
1.3k stars 147 forks source link

Replace build's distutils usage with setuptools #354

Closed ndjensen closed 1 year ago

ndjensen commented 2 years ago

When building Jep 4.0 with Python 3.10, we get the following warning:

DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils.core import setup, Extension

Philosophically, I had avoided depending on setuptools for the build because we didn't need it and I didn't want users to have to download an extra package/dependency. However, that philosophy is superseded by Python's philosophy/attitude toward distutils and setuptools. A nice thing about setuptools' independence from Python means it's not tied to the Python release schedule, so fixes and improvements can get in faster.

Switch the Jep build to use setuptools instead of distutils. Ideally this will be targeted for Jep 4.1 so we will have completed this ticket well before the release of Python 3.12.

ndjensen commented 2 years ago

Looks like setuptools includes all of distutils. So the quick fix may be to change the imports throughout the files in the jep/commands directory. A longer term fix may be to adapt Jep to a newer Python build system like the setup.cfg files.

ndjensen commented 2 years ago

Changing distutils imports to be setuptools._distutils did not work. Tried with Python 3.7 and setuptools 59.4.0.

$ python3 setup.py clean
numpy not found, running without numpy support
running clean
Traceback (most recent call last):
  File "setup.py", line 139, in <module>
    'test': test,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 148, in setup
    return run_commands(dist)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 984, in run_command
    cmd_obj = self.get_command_obj(command)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 859, in get_command_obj
    cmd_obj = self.command_obj[command] = klass(self)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/__init__.py", line 172, in __init__
    _Command.__init__(self, dist)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/cmd.py", line 57, in __init__
    raise TypeError("dist must be a Distribution instance")
TypeError: dist must be a Distribution instance
bsteffensmeier commented 2 years ago

I found some migration advice in PEP632 and the setuptools documentation. I was able to come up with something that can build, test, and install on python 3.10 on linux. A bunch of the utility stuff is still using setuptools._distutils but I tried to switch to setuptools wherever it was obvious what needed to be done. We have alot of windows specific code and I am certain I broke windows and it is likely I broke other platforms. We can use this as a starting point but I think we should get it tested and working on more platforms before committing and eventually we should try to move away from anything in _distutils if possible.

My change is here: https://github.com/bsteffensmeier/jep/commit/0a24ce5bdce1b77619db156473ecaec42473aea6

ndjensen commented 1 year ago

Closing as completed in Jep 4.1.0.