search

automl / amltk

A build-it-yourself AutoML Framework
https://automl.github.io/amltk/
BSD 3-Clause "New" or "Revised" License
68 stars 6 forks source link

[CI] Test hangs on Python 3.11 #257

Closed eddiebergman closed 9 months ago

eddiebergman commented 9 months ago

Not sure why exactly but on Python 3.11, see this job which had to be cancelled after 5 hours of not finishing... (normal test suite is <10 min)

It's hard to tell due to the fact it doesn't seem to say what test is currently running but this is the latest output that was available.

tests/scheduling/plugins/test_pynisher_plugin.py::test_trial_gets_autodetect_memory[dask_executor] PASSED [ 59%]
tests/scheduling/plugins/test_pynisher_plugin.py::test_trial_gets_autodetect_time[process_executor] PASSED [ 59%]
tests/scheduling/plugins/test_pynisher_plugin.py::test_trial_gets_autodetect_time[loky_executor] PASSED [ 59%]
tests/scheduling/plugins/test_pynisher_plugin.py::test_trial_gets_autodetect_time[dask_executor] PASSED [ 59%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_empty_kwargs_does_not_change_anything[process_executor] PASSED [ 59%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_empty_kwargs_does_not_change_anything[dask_executor] SKIPPEDs is that this is due to
the order in which imports are done when dask uses it's own unpikcling
strategy. It's rather undeterministic.)                                  [ 59%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_empty_kwargs_does_not_change_anything[sequential_executor] PASSED [ 59%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_empty_kwargs_does_not_change_anything[loky_executor] PASSED [ 59%]

Cross-referencing with the python 3.10 suite, it seems it might have gotten stuck here:

tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_empty_kwargs_does_not_change_anything[sequential_executor] PASSED [ 59%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_empty_kwargs_does_not_change_anything[loky_executor] PASSED [ 59%]
tests/scheduling/plugins/

########## This should have been the next test to run.
test_threadpoolctl_plugin.py::test_limiting_thread_count_limits_only_inside_task[process_executor] PASSED [ 59%]
###############

tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_limiting_thread_count_limits_only_inside_task[dask_executor] PASSED [ 60%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_limiting_thread_count_limits_only_inside_task[sequential_executor] PASSED [ 60%]
tests/scheduling/plugins/test_threadpoolctl_plugin.py::test_limiting_thread_count_limits_only_inside_task[loky_executor] PASSED [ 60%]

Will cancel and rerun the tests to see if this is a constant issue or a one off. This was done after #256 which seems to pass when not in master. Could be a flaky test which is problematic.

eddiebergman commented 9 months ago

Rerunning the tests seemed to work. Given that the original PR passed, I will close this but keep an eye on any re-occuring hangs.