Allow to install valkka-core with pip

[rnestler]

Currently the only way to get the Python bindings for valkka-core is to install valkka with apt. This makes it difficult to use it on other operating systems. Also when on Ubuntu it can't be used in a Python virtualenv.

[elsampsa]

Let's see.. Valkka has:

(1) library, i.e. the C++ part (2) Python bindings using (1)

Why would you want to install these two things separately? Then you would have to do: (1) with "apt-get" and after that (2) with "pip3 install".

Please keep in mind that (1) can not be installed with "pip3 install" in any case, for the same reasons that you can't install any media framework with "pip3 install". i.e., ffmpeg, vlc, gstreamer, etc.: too many dependencies (which can only be handled with apt) and too much of c(++) code to compile on-the-fly (during pip3 install which would take ages).

About "other operating systems": the library (1) only supports linux. It relies heavily on all things posix (shared memory, semaphores, etc.), so windows is out of question. In theory, it could maybe compile in mac, but I have never tried that.

If you need to use virtualenv, you could maybe manually enable individial packages that are installed globally (/usr/lib/python3/dist-packages was it..?), while disabling everything else..

For completely isolated installs, we do have docker images..

[rnestler]

About "other operating systems": the library (1) only supports linux.

I meant to say other distributions here which don't use apt.

Please keep in mind that (1) can not be installed with "pip3 install" in any case, for the same reasons that you can't install any media framework with "pip3 install". i.e., ffmpeg, vlc, gstreamer, etc.: too many dependencies (which can only be handled with apt) and too much of c(++) code to compile on-the-fly (during pip3 install which would take ages).

How do other large C++ project like Qt handle this? They seem to have prebuilt Qt for the manylinux1 ABI and bundle it in the wheel.

If you need to use virtualenv, you could maybe manually enable individial packages that are installed globally (/usr/lib/python3/dist-packages was it..?), while disabling everything else..

Just using mkvirtualenv --system-site-packages to create the virtualenv after installing the apt package did the trick for me.

[elsampsa]

Qt is sort of more self-contained - it doesn't depend on other shared libraries (i.e. ".so" files).

So they just include their compiled ".so" binary files into the python package.

In valkka, the build process produces two shared object .so files:

• libValkka.so.1.2.0 = the actual library (1)
• _valkka_core.so.1.2.0 = the python interface (produced by swig) (2)

Sure, we could include these in a pip package as-is. But this still leaves the "dangling" dependencies into (1)

Looking into "debian/control", these seem to be, at the moment:

python3, mesa-utils, glew-utils, python3-numpy, v4l-utils, python3-pip, openssl

hmm.. (why those utils-versions..?) this could be doable.. but then, depending on your distro, openssl, glew and v4l might be in different versions than what the libValkka.so was compiled for.. and then people start asking why valkka doesn't work, which means more (unnecessary and off-topic) work for me.

So yes, maybe this is doable, but not a priority in any way.

libValkka is also very easy to compile oneself if you don't want to use apt-get. Have you tried that?

[rnestler]

which means more (unnecessary and off-topic) work for me. So yes, maybe this is doable, but not a priority in any way.

I think this is a good reason not to invest too much time in it.

For me it was just a bit surprising that I can't pip install it in a virtualenv (even with having to install some system dependencies first) which I'm used to from other Python projects.

libValkka is also very easy to compile oneself if you don't want to use apt-get. Have you tried that?

Not yet and probably won't. I found the project via the https://elsampsa.github.io/valkka-examples/_build/html/qt_notes.html example since I have the same problem: Integrating a Qt UI with Python multiprocessing and wanted to take a look how it is done in valkka and check out the examples. So I'm only tangentially interested and don't need all the awesomeness of the valkka project for video processing :slightly_smiling_face:

I hope I didn't waste your time with this issue.

[elsampsa]

no problem!

If you do python multiprocessing, here is several years of experience in that subject. Highly recommended: :)

https://medium.com/@sampsa.riikonen/doing-python-multiprocessing-the-right-way-a54c1880e300

In the "frontend" (as defined in that article) then just send/receive Qt signals.

[elsampsa]

Here's the complete example! :)

https://github.com/elsampsa/medium/blob/main/multiprocess/example2.py