Support Python 3.11

[nzw0301]

Motivation

Sub-task of https://github.com/optuna/optuna/issues/3964 for tracking this repo.

Description

Add python 3.11 to github action's version matrix. An example PR is https://github.com/optuna/optuna-examples/pull/160.

Alternatives (optional)

Additional context (optional)

I summarise the status of targets as follows:

• [x] allennlp.yml
  • development has been stopped, we might not be able to support python 3.11
• [x] base.yml: https://github.com/optuna/optuna-examples/pull/160
• [x] catboost.yml
  • wait for https://github.com/catboost/catboost/issues/2213
• [x] chainer.yml https://github.com/optuna/optuna-examples/pull/163
• [x] checks.yml: fixed python 3.8
• [x] dashboard.yml
• [x] dask.yml https://github.com/optuna/optuna-examples/pull/163
• [x] dask_ml.yml
  • blocked by numba
• [x] fastai.yml
  • blocked by pytorch
• [x] haiku.yml
  • blocked by tensorflow
• [x] hiplot.yml https://github.com/optuna/optuna-examples/pull/163
• [x] hydra.yml https://github.com/optuna/optuna-examples/pull/160
• [x] keras.yml
  • blocked by tensorflow
• [x] lightgbm.yml
  • possibly not. https://github.com/microsoft/LightGBM/blob/master/.github/workflows/python_package.yml
• [x] mlflow.yml
  • blocked by numba
• [x] multi_objective.yml
  • blocked by pytorch
• [ ] mxnet.yml
  • it doesn't work at least my local env.
• [x] pytorch.yml
  • wait for https://github.com/pytorch/pytorch/issues/86566
• [x] ray.yml
  • https://github.com/optuna/optuna-examples/pull/174
• [x] rl.yml
  • blocked by pytorch
• [x] samplers.yml: https://github.com/optuna/optuna-examples/pull/160
• [x] skimage.yml
  • wait for https://github.com/scikit-image/scikit-image/issues/6517
• [x] sklearn.yml https://github.com/optuna/optuna-examples/pull/163
• [x] stale.yml fixed python 3.8
• [x] tensorboard.yml
  • blocked by tensorflow
• [x] tensorflow.yml
  • wait for https://github.com/tensorflow/tensorflow/issues/58032
• [x] tfkeras.yml
  • blocked by tensorflow
• [x] visualization.yml: https://github.com/optuna/optuna-examples/pull/160
• [x] wandb.yml: https://github.com/optuna/optuna-examples/pull/163
• [x] xgboost.yml https://github.com/optuna/optuna-examples/pull/163

[Kaushik-Iyer]

Are all of them done?

[nzw0301]

@Kaushik-Iyer As in the issue description, we need to wait for third party libraries' python 3.11 support.

[github-actions[bot]]

This issue has not seen any recent activity.

[github-actions[bot]]

This issue has not seen any recent activity.

[hvy]

tensorflow now supports 3.11, see https://github.com/optuna/optuna/issues/3964#issuecomment-1501446648

All examples blocked by tensorflow might be worth revisiting. While I haven't checked for possible errors, please let me mark this issue as contribution welcome for others to try picking it up. Please let me know if there are uncertainties.

[github-actions[bot]]

This issue has not seen any recent activity.

[github-actions[bot]]

This issue has not seen any recent activity.

[Alnusjaponica]

It seems Python 3.11 is now supported by tensorflow. See https://github.com/tensorflow/tensorflow/issues/58032 for more detail.

[Alnusjaponica]

Same as PyTorch(https://github.com/pytorch/pytorch/issues/86566) and Numba (https://github.com/numba/numba/issues/8304)

[github-actions[bot]]

This issue has not seen any recent activity.

[Alnusjaponica]

It seems the development of MXNet has stopped before the release of python 3.11 and the test does not pass. ~Therefore, it will be enough to close this issue after #214 is merged.~

[Alnusjaponica]

It is possible to run MXNet tests in Python 3.11, but it is necessary to install Numpy<1.24.0. To add to the Python 3.11 job to MXNet CI, reversing the order of the dependency list in mxnet/requirements.txt suffices. This is because Numpy>=1.24.0 is actually installed before running pip install -r requirements.txt in the CI, and it is necessary to reinstall Numpy<1.24.0 before the CI installs MXNet. For more details, please refer to https://github.com/Alnusjaponica/optuna-examples/pull/1.

[github-actions[bot]]

This issue has not seen any recent activity.

[nzw0301]

FYI: MXNet has been archived since 18 Nov. 2023, so the incompatibility with numpy will not be addressed anymore on the MXNet side.

[github-actions[bot]]

This issue has not seen any recent activity.

[not522]

All except MXNet have been supported, and MXNet has finished development. Therefore, Python 3.11 support is considered complete. Thank you for your contributions!