search

openfheorg / openfhe-python

Official Python wrapper for OpenFHE. Current release is v0.8.9 (released on September 10, 2024).
https://openfheorg.github.io/openfhe-python/html/index.html
BSD 2-Clause "Simplified" License
73 stars 21 forks source link

Add support for PyPI packaging. Fixes #59 #104

Closed Thadah closed 7 months ago

Thadah commented 8 months ago

Hello!

This PR adds support for PyPI packaging when using the command python setup.py sdist bdist_wheel. The purpose is to make easier for developers to create a package of their own and for the openfhe org to be able to publish packages easily.

I have created a setup.py that compiles the library with CMake and then creates the sdist tar.gz and bdist_wheel whl packages with the library inside, allowing for easy installation with pip install and integration with import openfhe.

There are some caveats to the approach I followed. Since build_ext executes at the end of the setup process, it was creating empty packages and then compiling the library. To work around that I created a custom sdist class that forces a build_ext before all the other commands. That introduces the problem of double compilation, which is done in the bdist_wheel phase. To prevent that, I created the (admittedly not very elegant) solution of passing the compilation if the .so file already exists. That's also why it gets removed during the sdist phase of the build. There also isn't any docstring for now, which complicated knowing which functions are available and what they do, forcing you to rely on the documentation.

If you have any feedback or a better approach to the code, please let me know!

Thank you.

Thadah commented 7 months ago

Sorry I just realized I was trying to push to main instead of dev. Fixed now.

yspolyakov commented 7 months ago

Adding @mtriplett-duality to this PR

mctriplett commented 7 months ago

Hi @Thadah Thanks so much for the work here! I've run this locally, and everything seems to work perfectly. Unless you have some other modifications to add, we're comfortable merging this in. Just give us the thumbs up!

Thadah commented 7 months ago

I found a way of making the pybind11 stubs show in most IDEs, but automating the process is a bit convoluted. It would require to create a wheel package, then installing it in a development (or virtual) environment, use pybind11-stubgen to generate the .pyi files, and generate the package again with the automatically generated pyi files. A bash script might be the best option.

Another way would be to create the .pyi files manually and place them in the openfhe/ directory so that they get packaged with the rest of the files. This would require periodic maintenance as they wouldn't be updated automatically and new functions or features would be lost to the IDE.

What approach would you prefer?

Thadah commented 7 months ago

I've added a build_package.sh script that should automate the stub creation and packaging automatically, but it will ask for the module to be installed first with pip install e .

I hope this is good enough.

Thadah commented 7 months ago

I believe there's nothing else to add for now on my part, so I have no issue for it to be merged. A CI/CD pipeline surely makes a lot more sense than doing workarounds in a setup.py file, but I wanted to provide a stopgap solution in the meantime. Thank you for all the feedback @mtriplett-duality!

Edit: I changed some lines in the shell script because I realized it wasn't working properly when using the project as a submodule and I wasn't looking if pybind11-stubgen was installed before running the script.