joblib / loky

Robust and reusable Executor for joblib
http://loky.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
528 stars 45 forks source link

MAINT bump-up CI versions #389

Closed ogrisel closed 1 year ago

ogrisel commented 1 year ago

Let's tests on more recent Python versions as well.

I removed CPython 3.7 from the matrix but we still test for Python 3.7 via PyPy. We would need to also bump it up but I think we can do that in a dedicated PyPy PR.

Note that testing with Python 3.11 might make the pytest orderning non-deterministic and might reveal old yet hidden bugs such as the one being investigated in #388.

ogrisel commented 1 year ago

I forgot to change the names in the CI config. Let me fix that now.

ogrisel commented 1 year ago

Hum, one of the windows failure on Python 3.11 looks like the failure we get under windows with the freeze support (#375) although the CI of the new failing test of #375 is running Python 3.10:

______________ TestsProcessPoolLokyExecutor.test_max_depth[True] ______________
loky.process_executor._RemoteTraceback: 
"""
loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "D:\a\1\s\loky\process_executor.py", line 426, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.2\x64\Lib\multiprocessing\queues.py", line 102, in get
    with self._rlock:
  File "C:\hostedtoolcache\windows\Python\3.11.2\x64\Lib\multiprocessing\synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.2\x64\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
"""

Still, they might be related problems.

tomMoral commented 1 year ago

Should we merge this one as this refects the state of the main branch and aim for fixes in followup PRs?