Can't install on Google Colab

[ericleonardo]

Please, how can Auto-PyTorch be installed on Google Colab? I tought it was related to Swig and already tried installing (!pip install swig==3.0.12) But error keeps the same when trying to install Auto-PyTorch: !pip install autopytorch --verbose

(...)

File "/tmp/pip-install-s9zzj7zb/scikit-learn_f1999172b8064fe2976e8b3e4e0cc343/sklearn/_build_utils/__init__.py", line 70, in cythonize_extensions
      config.ext_modules = cythonize(
    File "/tmp/pip-build-env-swfwaxz_/overlay/local/lib/python3.10/dist-packages/Cython/Build/Dependencies.py", line 1125, in cythonize
      result.get(99999)  # seconds
    File "/usr/lib/python3.10/multiprocessing/pool.py", line 774, in get
      raise self._value
  Cython.Compiler.Errors.CompileError: sklearn/ensemble/_hist_gradient_boosting/splitting.pyx
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/bin/python3 /usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpoinhxvvk
  cwd: /tmp/pip-install-s9zzj7zb/scikit-learn_f1999172b8064fe2976e8b3e4e0cc343
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[mlindauer]

@dengdifan could you please have a short look whether it is something obvious we can fix quickly? Seems to be a sklearn problem and not a problem with APT.

[dengdifan]

Hi @ericleonardo thanks for the issue, this is probably related to https://github.com/automl/auto-sklearn/issues/1670 You need to downgrade the python version in Colab to 3.9 could you try the following commands? (originally from https://stackoverflow.com/a/68530310)

#**Add python version you wish** to list
!sudo apt-get update -y
!sudo apt-get install python3.9
from IPython.display import clear_output 
clear_output()
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1

# Choose one of the given alternatives:
!sudo update-alternatives --config python3
# !sudo update-alternatives --config python

# Check the result
!python3 --version

# Attention: Install pip (... needed!)
!sudo apt install python3-pip
!sudo apt-get install python3.9-distutils
!python3.9 -m pip install swig==3.0.12
!pip install autopytorch --verbose

and check if AutoPyTorch can be installed and executed there? (Unfortunately, I was always disconnected from the Colab server and cannot check if AutoPyTorch is successfully installed there)

[ericleonardo]

Hi @dengdifan .. really Colab now keeps disconnecting after changing Python version. Installation seems successful, but sadly after restart, it is not possible to keep connected. Thank you

Successfully installed ConfigSpace-0.7.1 MarkupSafe-2.1.3 Pillow-10.0.0 PyWavelets-1.4.1 Shapely-2.0.1 absl-py-1.4.0 autopytorch-0.2.1 cachetools-5.3.1 catboost-1.2 certifi-2023.7.22 charset-normalizer-3.2.0 click-8.1.6 cloudpickle-2.2.1 cmake-3.27.1 contourpy-1.1.0 cycler-0.11.0 dask-2023.8.0 distributed-2023.8.0 emcee-3.1.4 filelock-3.12.2 flaky-3.7.0 fonttools-4.42.0 fsspec-2023.6.0 google-auth-2.22.0 google-auth-oauthlib-1.0.0 graphviz-0.20.1 grpcio-1.56.2 idna-3.4 imageio-2.31.1 imgaug-0.4.0 importlib-metadata-6.8.0 importlib-resources-6.0.1 jinja2-3.1.2 joblib-1.3.2 kiwisolver-1.4.4 lazy_loader-0.3 lightgbm-4.0.0 lit-16.0.6 locket-1.0.0 lockfile-0.12.2 markdown-3.4.4 matplotlib-3.7.2 more-itertools-10.1.0 mpmath-1.3.0 msgpack-1.0.5 networkx-3.1 numpy-1.25.2 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 oauthlib-3.2.2 opencv-python-4.8.0.76 packaging-23.1 pandas-2.0.3 partd-1.4.0 plotly-5.15.0 protobuf-4.24.0 psutil-5.9.5 pyasn1-0.5.0 pyasn1-modules-0.3.0 pynisher-0.6.4 pyparsing-3.0.9 pyrfr-0.8.3 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 scikit-image-0.21.0 scikit-learn-0.24.2 scipy-1.11.1 smac-1.4.0 sortedcontainers-2.4.0 sympy-1.12 tabulate-0.9.0 tblib-2.0.0 tenacity-8.2.2 tensorboard-2.14.0 tensorboard-data-server-0.7.1 threadpoolctl-3.2.0 tifffile-2023.7.18 toolz-0.12.0 torch-2.0.1 torchvision-0.15.2 tornado-6.3.2 triton-2.0.0 typing-extensions-4.7.1 tzdata-2023.3 urllib3-1.26.16 werkzeug-2.3.6 zict-3.0.0 zipp-3.16.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: The following packages were previously imported in this runtime:
  [certifi,cycler,dateutil,kiwisolver,psutil]
You must restart the runtime in order to use newly installed versions.

[Borda]

I see the same or similar issue when installing on Kaggle... :thinking: seem it comes from sklearn https://github.com/scikit-learn/scikit-learn/discussions/26910