vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.28k stars 590 forks source link

[FEATURE-REQUEST] Support Google Colab installation without runtime restart #2211

Closed franz101 closed 1 year ago

franz101 commented 2 years ago

Description When running pip install vaex, the runtime of the colab is needed to restarted (see attachments)

Is your feature request related to a problem? Please describe. The problem is some dependencies update matplotlib to larger then 3.22 (which runs on Google Colab) Additional context Here is a Google Colab that shows the problem and a workaround

Example of runtime restart needed:

Screenshot 2022-09-20 at 15 27 03

Example of installing the packages in a way matplotlib is not updated:

Screenshot 2022-09-20 at 15 27 56
franz101 commented 2 years ago

@maartenbreddels

JovanVeljanoski commented 2 years ago

I don't think we can do anything about that. It is their environment.

If somebody has an idea on how to fix this - please get in touch.

Edit: If you do not want to update or touch matplotlib in any way, just don't install vaex-viz. Install only vaex-core and perhaps vaex-hdf5 and vaex-ml depending on what you need.

maartenbreddels commented 2 years ago

Strange issue, thanks for letting us know.

Maybe by using the solution here https://github.com/geoalchemy/geoalchemy2/pull/392 in vaex (instead of the slow pkg_resources) we can make this faster and also working on colab without requiring a restart.

I think this could be a good first issue for someone.

franz101 commented 1 year ago

Hey @maartenbreddels ,

thanks for the input. I have forked vaex and am experimenting with the different version implementation: https://github.com/franz101/vaex-colab/blob/f70f4a4e2ef6c4d2902239e183f5d7abd4bcc4d3/packages/vaex-core/vaex/version.py

But running the installation it fails during the vaex-meta setup: https://colab.research.google.com/drive/1WC-ROEunPKaxHD50b_vqxHS_3FT9METM#scrollTo=apSprOWRQ2qy

is this related to python 3.7 and not a venv?

franz101 commented 1 year ago

Running setup.py develop for vaex-meta worked after cloning recursively

maartenbreddels commented 1 year ago

Yeah, we could improve there by giving a hint if the git submodules are empty.