search

bmcage / odes

The ODES scikit for ordinary differential and algebraic equations, an extension to scipy
https://scikits-odes.readthedocs.io/
Other
122 stars 44 forks source link

Upper-cap `setuptools` and remove support for Python `2.7`-`3.6` #137

Closed Saransh-cpp closed 2 years ago

Saransh-cpp commented 2 years ago

See https://github.com/numpy/numpy/issues/22135. Right now installing scikits.odes results in the following error -

Collecting scikits.odes
  Downloading scikits.odes-2.6.4.tar.gz (212 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.4/212.4 kB 55.0 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting scipy
  Downloading scipy-1.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.9/43.9 MB 49.4 MB/s eta 0:00:00
Collecting numpy<1.25.0,>=1.18.5
  Using cached numpy-1.23.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Building wheels for collected packages: scikits.odes
  Building wheel for scikits.odes (pyproject.toml): started
  Building wheel for scikits.odes (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  × Building wheel for scikits.odes (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      <string>:25: DeprecationWarning:

        `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
        of the deprecation of `distutils` itself. It will be removed for
        Python >= 3.12. For older Python versions it will remain present.
        It is recommended to use `setuptools < 60.0` for those Python versions.
        For more details, see:
          https://numpy.org/devdocs/reference/distutils_status_migration.html

      Traceback (most recent call last):
        File "/home/runner/work/PyBaMM/PyBaMM/.tox/coverage/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/home/runner/work/PyBaMM/PyBaMM/.tox/coverage/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/runner/work/PyBaMM/PyBaMM/.tox/coverage/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-5l28ns3f/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 412, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/tmp/pip-build-env-5l28ns3f/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-5l28ns3f/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 482, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-5l28ns3f/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 25, in <module>
        File "/tmp/pip-build-env-5l28ns3f/overlay/lib/python3.9/site-packages/numpy/distutils/core.py", line 24, in <module>
          from numpy.distutils.command import config, config_compiler, \
        File "/tmp/pip-build-env-5l28ns3f/overlay/lib/python3.9/site-packages/numpy/distutils/command/config.py", line [19](https://github.com/pybamm-team/PyBaMM/runs/7856676220?check_suite_focus=true#step:12:20), in <module>
          from numpy.distutils.mingw32ccompiler import generate_manifest
        File "/tmp/pip-build-env-5l[28](https://github.com/pybamm-team/PyBaMM/runs/7856676220?check_suite_focus=true#step:12:29)ns3f/overlay/lib/python3.9/site-packages/numpy/distutils/mingw[32](https://github.com/pybamm-team/PyBaMM/runs/7856676220?check_suite_focus=true#step:12:33)ccompiler.py", line 28, in <module>
          from distutils.msvccompiler import get_build_version as get_build_msvc_version
      ModuleNotFoundError: No module named 'distutils.msvccompiler'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for scikits.odes
Failed to build scikits.odes
ERROR: Could not build wheels for scikits.odes, which is required to install pyproject.toml-based projects

The following PR tries to use the latest version of NumPy - https://github.com/pybamm-team/PyBaMM/pull/2231

Saransh-cpp commented 2 years ago

Upper-capping setuptools<=64.0.0 works for me locally! 65.0.0 gives the error mentioned above and in the CI pipeline.

But, NumPy recommends hard pinning setuptools==59.2.0.

Maintainers' call!

Saransh-cpp commented 2 years ago

For more information -

PR that caused the breakage - https://github.com/pypa/setuptools/pull/3505 Issue that explains the breakage - https://github.com/pypa/setuptools/issues/3532

Saransh-cpp commented 2 years ago

This has been fixed in the new setuptools release - https://github.com/pypa/setuptools/compare/v65.0.1...main (reverts back the breaking change).

Saransh-cpp commented 2 years ago

I have decided to reopen this. Now NumPy advises pinning setuptools to the recommended version even more clearly and officially in their docs (especially for the projects using numpy.distutils). Follow https://github.com/numpy/numpy/issues/22135

See -

Either way, our recommendation is to pin setuptools to the last version that worked for you (59.x for numpy itself, maybe 64.0 for others). This is always a good idea to do in your releases. You don't want to find out that things work at release time, and then break afterwards because setuptools or another build-time dependency did a major release.

We already documented this at https://numpy.org/devdocs/user/depending_on_numpy.html#build-time-dependency, although we perhaps should make that a more prominent warning box. The relevant text there is: "For your other build dependencies you can probably be looser, however it’s still important to set lower and upper bounds for each dependency. It’s fine to specify either a range or a specific version for a dependency like wheel or setuptools. It’s recommended to set the upper bound of the range to the latest already released version of wheel and setuptools - this prevents future releases from breaking your packages on PyPI."

I will open a PR to make the warning more prominent, and will advertise the recommendation here again on the mailing list.

EDIT: note that there's a second place we recommend this, in the deprecation warning for numpy.distutils:

bmcage commented 2 years ago

@aragilar James, setuptools issues are more your cup of tea. Can you have a look?

Saransh-cpp commented 2 years ago

Changed the version to <=64.0.0 -

aragilar commented 2 years ago

I think if <=64 passes CI, then merging this makes sense. mesonpy's f2py support is still unclear last time I looked, so I don't think things are ready for to switch away from setuptools yet.

aragilar commented 2 years ago

Ok, based on the CI, it looks like a newer version of numpy than is supported is trying to be installed on python 2.7-3.6. I'd be inclined to drop those (3.5 and 3.6 are not longer getting security support), and merge this and do a new release so newer setuptools doesn't break building again.

Saransh-cpp commented 2 years ago

Thank you for the suggestions! I have removed the support for Python 2.7, 3.5, and 3.6. I think the CI should pass this time.

Saransh-cpp commented 2 years ago

A gentle bump - @bmcage, @aragilar! (The workflows require approval)

aragilar commented 2 years ago

Sorry, I'll try and sort this today.