OpenMined / PyDP

The Python Differential Privacy Library. Built on top of: https://github.com/google/differential-privacy
Apache License 2.0
491 stars 136 forks source link

Build wheel with Manylinux #473

Closed dvadym closed 9 months ago

dvadym commented 9 months ago

This PR implements wheel build with Manylinux for Linux.

Why is it needed?

When a Linux binary is built , it is linked against the standard libraries (libc, libstdc++ etc) of versions that are present on the built machine. Those libraries are forward compatible, namely it's possible to run the binary on machines which have standard libraries newer than ones against which this binary was built. But it's not backward compatible, i.e. the binary might fail on the machines with older libraries with errors like [usr/lib/libstdc++.so.6: version GLIBCXX_3.4.29 not found].

Currently since PyDP is built on Ubuntu 22, it fails to run on many older machines. We can choose Ubuntu 20, but there will be still problems with older machines.

What is the solution?

The common solution for Python packages which have C++ parts are built C++ with Manylinux docker images. Those are Docker images which contain old enough standard libraries, which improves compatibility.

For building cibuildwheel is used. This libraries manages build in Docker with Manylinux images.

Note: many common Python libs (e.g. Numpy, Matplolib) use cibuildwheel. Here) are some other examples.

What in this PR

It contains extending publish workflow with build on manylinux for Linux platform. Other platforms are not changed. Namely it contains

  1. Introducing build_PyDP_linux.sh which installs bazel and run build_PyDP.sh.
  2. Extending setup.py to call build_PyDP_linux.sh for Linux.
  3. Updating publish.yml to use cibuildwheel on Linux