deepof 0.7 Installation issue

[maRce10]

I tried installing deepof in google colab when running the preprocessing tutorial but got many conflicts:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. albucore 0.0.13 requires numpy<2,>=1.24.4, but you have numpy 1.23.5 which is incompatible. albumentations 1.4.13 requires numpy>=1.24.4, but you have numpy 1.23.5 which is incompatible. albumentations 1.4.13 requires scipy>=1.10.0, but you have scipy 1.8.1 which is incompatible. arviz 0.18.0 requires scipy>=1.9.0, but you have scipy 1.8.1 which is incompatible. bigframes 1.14.0 requires scikit-learn>=1.2.2, but you have scikit-learn 1.2.0 which is incompatible. bokeh 3.4.3 requires tornado>=6.2, but you have tornado 6.1 which is incompatible. chex 0.1.86 requires numpy>=1.24.1, but you have numpy 1.23.5 which is incompatible. cudf-cu12 24.4.1 requires numba>=0.57, but you have numba 0.56.4 which is incompatible. cudf-cu12 24.4.1 requires pandas<2.2.2dev0,>=2.0, but you have pandas 1.5.3 which is incompatible. distributed 2024.7.1 requires dask==2024.7.1, but you have dask 2024.5.1 which is incompatible. flax 0.8.4 requires jax>=0.4.19, but you have jax 0.4.18 which is incompatible. google-colab 1.0.0 requires ipykernel==5.5.6, but you have ipykernel 6.29.5 which is incompatible. google-colab 1.0.0 requires pandas==2.1.4, but you have pandas 1.5.3 which is incompatible. google-colab 1.0.0 requires tornado==6.3.3, but you have tornado 6.1 which is incompatible. inflect 7.3.1 requires typeguard>=4.0.1, but you have typeguard 2.13.3 which is incompatible. jaxlib 0.4.26+cuda12.cudnn89 requires scipy>=1.9, but you have scipy 1.8.1 which is incompatible. orbax-checkpoint 0.5.23 requires jax>=0.4.26, but you have jax 0.4.18 which is incompatible. pandas-stubs 2.1.4.231227 requires numpy>=1.26.0; python_version < "3.13", but you have numpy 1.23.5 which is incompatible. rmm-cu12 24.4.0 requires numba>=0.57, but you have numba 0.56.4 which is incompatible. scikit-image 0.23.2 requires scipy>=1.9, but you have scipy 1.8.1 which is incompatible. tf-keras 2.17.0 requires tensorflow<2.18,>=2.17, but you have tensorflow 2.12.0 which is incompatible. torchaudio 2.3.1+cu121 requires torch==2.3.1, but you have torch 2.2.2 which is incompatible. torchtext 0.18.0 requires torch>=2.3.0, but you have torch 2.2.2 which is incompatible. xarray 2024.6.0 requires pandas>=2.0, but you have pandas 1.5.3 which is incompatible.

I tried fixing them but more come up. Is there something I am missing?

thanks

[NoCreativeIdeaForGoodUserName]

Hi maRce10,

Thank you for using deepof. I'll try to recreate this issue tomorrow. The problem is likely that some package that deepof uses was updated which then can lead to dependency conflicts down the line.

Best regards, Patrick

[NoCreativeIdeaForGoodUserName]

Hi maRce10,

I replicated the issue and while deepof 0.7.0 runs fine in a local virtual environment with jupyter lab we get all kinds of conflicts with google colab. This is likely because google colab does not provide a true "clean" virtual environment as it comes with literally hundreds of pre-installed packages. Hence installing deepof in Colab leads to a lot of version conflicts. I will talk to a colleague tomorrow to find out the best way to solve the problem.

Best regards, Patrick

[maRce10]

Awesome! thanks

[NoCreativeIdeaForGoodUserName]

Hi, it has been a while. We have not yet decided how to best solve this issue but I wanted to at least give you a short update:

As expected, the issue comes from Google enforcing the pre-installation of hundreds of packages that are in conflict with the deepof packages. As google also does not allow for the setup of a clean virtual environment, there is no easy solution to fix these conflicts. I also want to add that this is not because "Google is stupid", it comes from the design idea of providing the users with everything they may possibly need already preinstalled in Colab. This is all good and fine as long as you do not need any more complicated additional packages (like deepof). To resove this we are currently following a double tracked approach: One is to find some kind of workaround to get deepof to run in Colab again. The other is finding an alternative to Colab that does not enforce pre-installation of packages in virtual environments.

Best regards, Patrick

[maRce10]

It makes sense. thanks for the update!

[NoCreativeIdeaForGoodUserName]

Final update on this issue:

We looked into multiple alternatives to Colab and it seems like either the environment is set up similarly to Colab in the sense that packages are preinstalled and you cannot create a truly empty virtual environment (e.g. Kaggle Kernels), they have this option but are barely used by anyone (e.g. Binder) or are not really suitable for other reasons. Respectively, we will drop the support for Google Colab and are going to refer users to Jupyter Notebook with the next release instead. To use deepof, you only need to be able to set up a clean virtual environment before installing deepof via pip. This is only a feature that is either blocked or made unnecessarily difficult in many Colab alternatives.