deephaven / deephaven-core

Deephaven Community Core
Other
238 stars 79 forks source link

Python Ticking Client: make wheels for supported architectures so the user doesn't have to build themselves #4130

Open kosak opened 1 year ago

kosak commented 1 year ago

It would be nice to be able to provide wheels for the pydeephaven_ticking package so that people don't have to go through the build process themselves.

jcferretti commented 7 months ago

A discussion recently resurfaced in slack about this. https://illumon.slack.com/archives/G01K6UHJ00K/p1701272435038839?thread_ts=1698167133.022949&cid=G01K6UHJ00K I didn't mention it there, but thinking more carefully about it, I think two changes Corey has either already made or is in the process of making for the C++ client may remove most of the roadblocks for this.

  1. Corey already removed python dependencies for any gRPC code in the DH client; the way the cython integration works, it depends on native code that only knows how to process barrage updates, and to actually get the updates it depends on the python layer (and pyarrow's version of flight) to handle the gRPC-ing. This means that what the python module needs in terms of native shared libraries is much smaller and does not pull in / has dependencies to the shared libraries for libgrpc, libprotobuf, etc.
  2. Corey is in the process of removing dependencies to POSIX calls (and any other unix/linux only libraries) from the C++ code as part of the effort to make the C++ client build on Windows (separate PR, unrelated to this issue). By removing the dependency to POSIX calls that would also clean up any potential entanglement with linux distribution specific libraries (the way he is doing it is to lean on some other libraries for, eg, time parsing, and depend on those libraries via "vendoring", which is exactly what the python wheel packaging documentation suggests doing: https://pypackaging-native.github.io/key-issues/native-dependencies/).

With those two things out of the way, we may be in a much better position now to tackle wheel building for python. Sadly, this doesn't improve the R or C++ itself situation a lot, since only python benefits from (1) above.

rbasralian commented 3 months ago

5267 covers this for Linux (which was the priority).

We still need to cover MacOS and Windows, eventually.