nens / threedigrid-builder

Generate a 3Di simulation grid from a model schematisation.
Other
1 stars 0 forks source link

Maintenance: Github actions test matrix must be updated #348

Closed caspervdw closed 2 months ago

caspervdw commented 5 months ago

The current test matrix tests with Python version 3.7 up to 3.11, corresponding to package versions in the years 2018-2022

margrietpalm commented 5 months ago

Running the tests with Python 3.12 with numpy fails.

Collecting numpy==1.24.*
  Downloading numpy-1.24.4.tar.gz (10.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.9/10.9 MB 108.2 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/importlib/__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1[36](https://github.com/nens/threedigrid-builder/actions/runs/7912954928/job/21599646276#step:6:37)0, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1[38](https://github.com/nens/threedigrid-builder/actions/runs/7912954928/job/21599646276#step:6:39)7, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 995, in exec_module
        File "<frozen importlib._bootstrap>", line [48](https://github.com/nens/threedigrid-builder/actions/runs/7912954928/job/21599646276#step:6:49)8, in _call_with_frames_removed
        File "/tmp/pip-build-env-q0y6q96e/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
          import setuptools.version
        File "/tmp/pip-build-env-q0y6q96e/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
          import pkg_resources
        File "/tmp/pip-build-env-q0y6q96e/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 21[72](https://github.com/nens/threedigrid-builder/actions/runs/7912954928/job/21599646276#step:6:73), in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Error: Process completed with exit code 1.

This is a known issue with python 3.12: https://github.com/numpy/numpy/issues/23808

This can be fixed by upgrading numpy to 1.26. and setuptools to 69.. However, building the fortran part fails with those dependencies (without clear errors during building):

Run pytest
ImportError while loading conftest '/home/runner/work/threedigrid-builder/threedigrid-builder/threedigrid_builder/tests/conftest.py'.
threedigrid_builder/__init__.py:1: in <module>
    from .application import *  # NOQA
threedigrid_builder/application.py:17: in <module>
    from threedigrid_builder.grid import Grid, QuadTree
threedigrid_builder/grid/__init__.py:22: in <module>
    from .quadtree import *  # NOQA
threedigrid_builder/grid/quadtree.py:[11](https://github.com/nens/threedigrid-builder/actions/runs/7913597226/job/21601581991#step:7:12): in <module>
    from .fgrid import m_cells, m_quadtree
threedigrid_builder/grid/fgrid/__init__.py:1: in <module>
    from ._fgrid import m_cells, m_quadtree  # NOQA
E   ModuleNotFoundError: No module named 'threedigrid_builder.grid.fgrid._fgrid'
Error: Process completed with exit code 4.

Seems that the installer no longer supports what we do:

Running command python setup.py bdist_wheel
  /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
  !!
          ********************************************************************************
          Requirements should be satisfied by a PEP 517 installer.
          If you are using pip, you can try `pip install --use-pep517`.
          ********************************************************************************
  !!
    dist.fetch_build_eggs(dist.setup_requires)

https://github.com/nens/threedigrid-builder/actions/runs/7916215007/job/21609611018

caspervdw commented 5 months ago

Tagging @martijn-siemerink and @daanvaningen because they know how the f2py build works.

daanvaningen commented 5 months ago

Looks fun! We will discuss who is going to pick this up :)

daanvaningen commented 4 months ago

same action required as https://app.zenhub.com/workspaces/team-3di-5ef60eff1973dd0024268b90/issues/gh/nens/threedigrid-builder/257