search

aperture-data / aperturedb-python

ApertureDB Python Client
Other
7 stars 3 forks source link

Dask installation failing on newest package #499

Closed drewaogle closed 2 hours ago

drewaogle commented 1 month ago

When working on a change for athena, pip started failing. I traced the error to pip selecting an old dask ( 0.8, I think ) which failed to install because it used a word that became reserved in python 3.7.

Note - this fails only with updated pip (24.2)

pip3 install     dask ffmpeg-python     opencv-python     pandas==1.5.3     numpy     dbeval     requests     boto3     jsonschema    aperturedb==0.4.33 pre-commit

is the line that fails.

pip3 install     dask ffmpeg-python     opencv-python     pandas==1.5.3     numpy     dbeval     requests     boto3     jsonschema    aperturedb==0.4.32 pre-commit

works

but also - 

pip3 install     dask>=2022.10.2 ffmpeg-python     opencv-python     pandas==1.5.3     numpy     dbeval     requests     boto3     jsonschema    aperturedb==0.4.33 pre-commit

works.

The fact that I was able to nudge dask to choose a newer version and it then selected the newest makes me think it is a problem with the pip resolver, which has had known issues in the past. Additionally, when you run the top, it shows that pip ends up panicking and searching through a lot of dasks for some reason.

drewaogle commented 1 month ago

Also if you run it in a vm, you can notice the pip process maxing cpu - which shouldn't be happening, and this is when it is trying to determine the "best" version.

gsaluja9 commented 3 hours ago

Where is this combination restricting. Especially the exact pandas version == 1.5.3

This does not repro in our CI, so lets add some more light to it.