search

datoviz / datoviz

⚡ Datoviz: high-performance GPU scientific data visualization C/C++/Python library
https://datoviz.org
MIT License
598 stars 25 forks source link

Python bindings #13

Open DataExplorerUser opened 3 years ago

DataExplorerUser commented 3 years ago

Hi, am I understanding the documentation correctly that a user has to compile Datoviz before using it with Python? If so, what would be a rough estimate when Datoviz would be available in an more accessible manner, e.g. pip install or otherwise?

rossant commented 3 years ago

Yes, you're correct! I'm considering making a first release (pip installable) in May 2021 but no guarantee yet.

DataExplorerUser commented 3 years ago

I appreciate it's open source. Would be very interesting to be able to use when it is available!

djhoese commented 3 years ago

Related to this, @rossant do you think it would be possible to get the Vulkan SDK and all other low-level dependencies on conda-forge? Or are those too low-level and too difficult to build for each platform that it would just be better to have the user install them outside of conda? If we could make datoviz conda installable with all dependencies that would be amazing.

rossant commented 3 years ago

In principle (should be double-checked), the Vulkan SDK is only required when compiling the library. Since all users need to compile Datoviz for now, they need the SDK. Otherwise, the only thing they'd need is a Vulkan-compatible graphics driver.

Once we build a binary package, either with pip or conda, the Python package will come with the libdatoviz shared library compiled for a given platform (.so/.dll/.dylib), as well as the Cython compiled binding module (pxd/pyx files). The shared library will also have some dynamic dependencies (especially if it was compiled with ffmpeg support, which is totally optional). Depending on the platform, we'll have to make sure these dependencies match between the compilation platform and the target user platform.

In practice, the compiled Datoviz library has few dependencies. Most dependencies (eg the Vulkan SDK, glfw3, etc) are statically linked at compile time, so they are bundled into the shared library and they don't need to be installed by the user. ffmpeg is the main exception, it needs to be handled separately.

Long story short, making robust cross-platform binary packages of Datoviz that are pip- and conda- installable with no effort from the user is definitely possible, it requires some work by the developers and package managers, but not by the end-user. Once the library is a bit more mature, the goal is definitely to have a Python library that is as easy to install as NumPy/SciPy/matplotlib etc (who have similar issues with a lot of low-level code in C and other languages).

In conda, perhaps it would make sense to have two separate packages: one that isn't Python-specific (libdatoviz?) and that could be reused by other wrappers in different languages, and one with just the Cython bindings. At the moment the Cython module uses dynamic linking (I initially tried static linking but I had some issues).

I have no experience with conda-forge but it would be awesome if it was able to build binary packages for all platforms automatically.

awesome-developer-20 commented 3 years ago

I'm considering making a first release (pip installable) in May 2021 but no guarantee yet.

Any updates on this?

rossant commented 3 years ago

There are highly experimental and preliminary Python wheels here. The wheel creation pipeline needs more work and testing and any help would be highly appreciated.

frozenranger commented 3 years ago

If possible can you make a python 3.7 whl? It was a struggle to build it manually with cmake / mingw lol. Thanks for your hard work sir! :) I will try to make a basic real-time plotting example this week based off your eventloop