search

microsoft / DiskANN

Graph-structured Indices for Scalable, Fast, Fresh and Filtered Approximate Nearest Neighbor Search
Other
1.02k stars 208 forks source link

[Question] Why require numpy version stick to 1.25? #544

Open KuribohG opened 4 months ago

KuribohG commented 4 months ago

I want to build diskannpy for py3.8, however I found this PR: https://github.com/microsoft/DiskANN/pull/396. Why diskannpy relies on exactly version 1.25 of numpy? I didn't find further explanation in this repo. Seems this PR is related to subnormal float computation. Did numpy 1.25 has some special features or bug fixes for subnormal floats?

daxpryce commented 3 months ago

@KuribohG sorry I'm so late in getting back to you, I just now saw this issue.

There is no reason that you need to stick with numpy 1.25. The problem is that the version that is compiled is compiled against the headers in numpy - and when I published to pypi I could pick precisely one version to do that for.

For further flexibility in this regard, we would need to create a conda recipe to build this. I didn't have time to do this myself, though I'd be eager for a PR in this regard if anyone could make it.

If you want to build against a different version of numpy, you are more than welcome to! To do this, follow these steps (note they are probably rough as they are from my flawed memory, but it might be enough to get you started)

1: Install all the project's prerequisites for your system. The instructions are in the README.md in the root of the repo. If you're building for ubuntu, there's a helper script in scripts/dev/install-dev-deps-ubuntu.bash that will do it all at once. 2: Edit pyproject.toml and change the version to 1.26, 1.24, 1.23 etc. 3: create a virtual environment python3.11 -m venv venv and activate it source venv/bin/activate 4: pip install build 5: python -m build

This should create a wheel that will work with your chosen version of numpy.

Note that this is ultimately what a conda recipe would do.

daxpryce commented 3 months ago

Also note: you can use whatever python version you'd like, >= 3.9