Python 3.12 Compatibility

[scheuerdo]

PFN's are incompatible with Python 3.12

Hey, I just updated to Python 3.12 and found out, that PFN's are incompatible due to the dependency requirement 'scikit-learn>=0.24.2,<1.2', in pyproject.toml.

Thrown Error

In detail, when installing PFN's with pip install -e . the following error is thrown:

Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/dominik/PFNs
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting bayesmark>=0.0.8 (from pfns==0.1.5)
  Using cached bayesmark-0.0.8-py3-none-any.whl
Collecting botorch>=0.7.2 (from pfns==0.1.5)
  Using cached botorch-0.11.0-py3-none-any.whl.metadata (11 kB)
Collecting configspace>=0.4.21 (from pfns==0.1.5)
  Using cached ConfigSpace-0.7.1-cp312-cp312-linux_x86_64.whl
Collecting gpytorch>=1.5.0 (from pfns==0.1.5)
  Using cached gpytorch-1.11-py3-none-any.whl.metadata (7.4 kB)
Requirement already satisfied: numpy>=1.21.2 in /home/dominik/.local/lib/python3.12/site-packages (from pfns==0.1.5) (1.26.4)
Requirement already satisfied: pyyaml>=5.4.1 in /usr/lib64/python3.12/site-packages (from pfns==0.1.5) (6.0.1)
Requirement already satisfied: requests>=2.23.0 in /usr/lib/python3.12/site-packages (from pfns==0.1.5) (2.31.0)
Collecting scikit-learn<1.2,>=0.24.2 (from pfns==0.1.5)
  Using cached scikit-learn-1.1.3.tar.gz (7.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 995, in exec_module
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "/tmp/pip-build-env-0y35hh53/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
          import setuptools.version
        File "/tmp/pip-build-env-0y35hh53/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
          import pkg_resources
        File "/tmp/pip-build-env-0y35hh53/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Reason

The reason for the error is that the long-deprecated pkgutil.ImpImporter class was removed with Python 3.12 (see Stackoverflow Issue) and for me, Scikit-learn version 1.2.2 worked but then another error is obviously thrown because of the requirement of scikit-learn < 1.2. Python 3.12 has been officially supported by scikit-learn since version 1.3.1.

[SamuelGabriel]

Ok, I updated our README to reflect that it does not work for py>=3.12, did you work on some code to fix the dependency issue? Otherwise, I think I can have a look this week.

[scheuerdo]

No, sorry. I didn't have time to look deeper into that, so I just switched to Python 3.9 for my thesis project.