search

microsoft / MLOS

MLOS is a project to enable autotuning for systems.
https://microsoft.github.io/MLOS
MIT License
136 stars 63 forks source link

distutils deprecation with Python 12 preventing mlos_bench from launching #865

Closed eujing closed 32 minutes ago

eujing commented 2 hours ago

With docker images built on the image python:3.12-slim, mlos_bench (or even the storage API) now fails to run due to the deprecation of distutils

File "/root/.local/lib/python3.12/site-packages/dashboard/main.py", line 5, in <module>
from mlos_bench.storage import Storage
File "/root/.local/lib/python3.12/site-packages/mlos_bench/storage/__init__.py", line 7, in <module>
from mlos_bench.storage.base_storage import Storage
File "/root/.local/lib/python3.12/site-packages/mlos_bench/storage/base_storage.py", line 19, in <module>
from mlos_bench.storage.base_experiment_data import ExperimentData
File "/root/.local/lib/python3.12/site-packages/mlos_bench/storage/base_experiment_data.py", line 8, in <module>
from distutils.util import strtobool  # pylint: disable=deprecated-module
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'

The workaround is to explicitly pip install setuptools to provide distutils, but maybe we want to consider moving away from disutils?

eujing commented 2 hours ago

@bpkroth @motus

eujing commented 2 hours ago

I have not looked too deeply into the overall dependency of distutils yet, but if we only need strtobool the official implementation seems short enough to copy into the codebase.

https://github.com/python/cpython/blob/0c5fc27217525c4e40b4064e6979f467540c2fc8/Lib/distutils/util.py#L308

bpkroth commented 2 hours ago

We test python-3.12 in the pipelines, so not sure what's different here, but I agree, we can just copy that into the one of the utils modules. Do you want to make a PR or shall I?