lmcinnes / umap

Uniform Manifold Approximation and Projection
BSD 3-Clause "New" or "Revised" License
7.46k stars 809 forks source link

umap 0.5.4 is causing a broken package error in Conda feedstock builds #1054

Closed RogerHYang closed 1 year ago

RogerHYang commented 1 year ago

With UMAP 0.5.4 we're seeing a broken package error in our Conda feedstock builds

0.5.4: broken package

Screenshot 2023-09-20 at 10 03 33 AM

0.5.3: no problem

Screenshot 2023-09-20 at 10 56 25 AM
lmcinnes commented 1 year ago

We actually weakened/removed requirements on tbb. It also seems odd since it looks like it is installing tbb via conda, so I really don't quite know what is going astray with pip check. I admit this is into weeds of python packaging that I am less familiar with. Any suggestions of what to actually look at would be appreciated.

hamelin commented 1 year ago

We have dropped reliance on package TBB on non-Intel platform, so as to support installing on ARM-based Apple computers. Could you please give more information on the platform where you observe this error?

RogerHYang commented 1 year ago

Could you please give more information on the platform where you observe this error?

The error is from our Conda feedstock build, and the log can be seen here. it seems to be running on a linux 64 system

{
  "linux_64_": {
    "CONFIG": "linux_64_",
    "UPLOAD_PACKAGES": "True",
    "DOCKER_IMAGE": "quay.io/condaforge/linux-anvil-cos7-x86_64"
  }
}
hamelin commented 1 year ago

Good news: I can duplicate the problem in a local environment. The bad news: this seems to be because the list of packages that Pip sees is a subset of what gets installed. To wit, conda list gives something close to what @RogerHYang reported:

image

Then pip list is less profligate:

image

This is likely why pip check fails: if it cannot see tbb in its package listing, it cannot possibly figure out the proper dependency structure of umap-learn.

I will investigate this behavior further. However, @RogerHYang, I ask out of curiosity: what is the purpose of checking dependency resolution with pip check when all the dependencies are expected to be resolved from the Conda-Forge package set?

hamelin commented 1 year ago

Also, to be clear: this doesn't seem to be a direct UMAP problem, but rather a brittle behavior from the mixture of Conda and Pip.

RogerHYang commented 1 year ago

Thank you for the analysis @hamelin. This is new to me and I used grayskull to generate the meta.yaml that has the pip check, which is in fact unnecessary as you pointed out. I'll mark this issue as resolved.