`--without-libspotify` does not seem to work

[CapOM]

Steps on Ubuntu 18.04:

• tizonia-dev-build --deps
• sudo apt-get remove libspotify12 libspotify-dev
• tizonia-dev-build --no-player --debug --install --without-libspotify

Output:

checking for SPOTIFY... no
configure: error: Package requirements (libspotify >= 12.1.51) were not met:
No package 'libspotify' found
configure: error: ./configure failed for spotify_source

The second step is too simulate the fact that it seems that libspotify is not packaged in Fedora. In any case it seems that --without-libspotify has regressed and does not work anymore.

We are trying to enable Tizonia https://gitlab.freedesktop.org/gstreamer/gst-ci/issues/20 in the automated builds of gst-omx (GStreamer OpenMAX)

[tizonia]

Hi Julien!

The tizonia-dev-build script does not yet support an option to remove libspotify (only to remove the 'player' feature).

Have you tried without the script (i.e. the traditional autoreconf -ifs + configure + make + make install)?

./configure --disable-player --without-libspotify

[CapOM]

Thx Juan, indeed that worked: autoreconf -ifs && ./configure --disable-player --without-libspotify && make && make install

[CapOM]

So in order to reduce build time in our CI, it would be great to add more options:

• an option to disable all clients
• an option to enable a list of plugins, for example--enable-plugins=mp3_decoder,mp3_encoder,aac_decoder,vp8_decoder,vorbis_decoder,opus_decoder,flac_decoder

In theory just autoconf and build tiz core and the plugins should be something quick.

(Maybe when moving to meson ?)

Thx!

[lgbaldoni]

That should be trivial on meson, but if implemented the latter would no longer be aligned with autotools.

[tizonia]

@CapOM

Thanks to @lgbaldoni , Tizonia is about to gain a shiny new meson build system!. The new build system is already present in the 'develop' branch. I would like to cut a new release of Tizonia very soon, maybe this week or next. It will ship with both Autotools and Meson build systems. Meson will be in 'preview' mode, with the hope to obsolete autotools in the next cycle.

--enable-plugins=mp3_decoder,mp3_encoder,aac_decoder,vp8_decoder,vorbis_decoder,opus_decoder,flac_decoder

Regarding this, I think I would rather have a --disable-plugins option instead, but perhaps meson could do both without much of a fuss?

In any case, I would not mind at all if the autotools build system is left a bit behind in terms of features. At the end of the day we all want to get rid of it sometime soon :-)

[CapOM]

Sounds like a good plan, thx!

[lgbaldoni]

See PR https://github.com/tizonia/tizonia-openmax-il/pull/671. Most likely broken in some combinations, please let me know if there's something to fix.

EDIT: -Dclients=false -Dplugins='' to disable all clients and plugins. To enable only specific plugins, list them separated by commas.

[tizonia]

@lgbaldoni

Thanks, I've merged the PR. I will be doing some testing around valid/invalid combinations later this week. We'll take it from there in case there are any broken combinations.

[gdesmott]

Thanks for the patch.

I gave it a quick shot and it still requires boost-python3, liboggz and libmediainfo. Is that really required for a minimal build?

Also, should I still define TIZONIA_REPO_DIR and TIZONIA_INSTALL_DIR or is it handled by meson properties now?

[lgbaldoni]

@tizonia sorry for the delay and for the suboptimal results. Unfortunately (or otherwise) meson is not Turing-complete and one can't play with arrays as it would be needed in this case.

@gdesmott right, I forgot to check for those. Will try and refine the logic if Juan doesn't beat me to it.

Regarding those variables, I think they are related to the debian package creation, which I didn't test at all. Still, every file is being installed in the same position as the autotools original, so I'm hopeful.

[tizonia]

@gdesmott / @lgbaldoni

Thanks for the quick feedback!

I haven't had a chance to try it yet, but yes, boost-python3, liboggz and libmediainfo would not be needed for a minimal build.

I think it is probably a good idea to define now what a minimal build would look like. I think the minimal build should take care of building the OpenMAX IL infrastructure, which amounts to building these directories:

3rdparty -> the D-BUS library used by the OMX IL Resource Management server
include -> the OMX IL headers
libtizcore -> the OMX IL Core
libtizonia -> the OMX IL base component/plugin framework
libtizplatform -> the OS support library
rm -> the OMX IL Resource Management logic

These directories form the minimal OpenMAX IL infrastructure that would be needed by a third-party project interested in OpenMAX IL. On top of that, a third-party project could optionally add a list of plugins + clients.

Regarding 'clients', they are dependencies of some of the plugins, but to simplify things, we may want to assume independence for now, and perhaps just document the dependencies so that a user can at least issue on the command-line the right combination of clients+plugins that will work.

I make a note to document these dependencies between clients and plugins so that we at a minimum can have that documented. Maybe later we could make meson implement this dependency chain if it's not too much work.

Regarding TIZONIA_REPO_DIR and TIZONIA_INSTALL_DIR, they are not needed for meson. These variables are only used by the convenience shell scripts that I created to simplify the build process with autotools and for various debian packaging needs.

[lgbaldoni]

@gdesmott https://github.com/tizonia/tizonia-openmax-il/pull/673. libmediainfo is required by the player.

[tizonia]

@gdesmott / @lgbaldoni

I've just merged #673

[gdesmott]

Thanks.

So I can now configure using meson build -Dclients=false -Dplugins='' but building seems to get stuck in some infinite loop. On my laptop it ended up killed by OOM and it's failing in a same way in my CI job, see https://gitlab.freedesktop.org/gdesmott/gst-ci/-/jobs/1505352

[CapOM]

(just something I spotted: it looks like the player is still being built: [163/181] Compiling C++ object 'player/src/431b6f0@@tizonia@exe/services_tunein_tiztuneinmgr.cpp.o')

[lgbaldoni]

@gdesmott /usr/bin/bash: line 109: cd: gst-build/: No such file or directory in your build log. Regarding your laptop, I suggest you to run ninja with -j1: since the player is a beast.

@CapOM you also need to pass -Dplayer=false if you don't want it to be built. I suppose clients and plugins are hard functional dependencies for it, but not build ones.

[gdesmott]

@gdesmott /usr/bin/bash: line 109: cd: gst-build/: No such file or directory in your build log.

Yes, that's because the build failed. The interesting part is:

virtual memory exhausted: Cannot allocate memory

Regarding your laptop, I suggest you to run ninja with -j1: since the player is a beast.

Indeed building with -j1 works but takes a very long time: 30 minutes on the CI which is way too much as this will likely become the bottleneck of our whole CI pipeline. Is there any other way we could speed up the process by disabling more components?

[CapOM]

@gdesmott The player is not needed: -Dplayer=false Should allow to use -j N

[lgbaldoni]

Indeed building with -j1 works but takes a very long time: 30 minutes on the CI which is way too much as this will likely become the bottleneck of our whole CI pipeline. Is there any other way we could speed up the process by disabling more components?

You should really ask Juan about that. Apparently the player bulkiness comes from Boost and it's really difficult to shake off. This said, do you actually need it?

[gdesmott]

Indeed, disabling the player works just fine for the CI! Thanks guys :)

[tizonia]

I'm glad that you guys can now build faster in your CI.

Basically, everything said is correct. The player is an end-user application that uses of the OMX IL infrastructure, so it is not meant to be reused. And yes the player build process is a pain for the compiler due to the boost meta state machine library that it's used there.

The plugins are discovered at run-time, so no build dependencies in general. There is one exception though. With Autotools, the player's Spotify logic is disabled when the Spotify plugin is not included in the build. This exception was included so that the player could be built in OpenSUSE (for some reason libspotify was/is not available there): https://build.opensuse.org/package/view_file/home:malcolmlewis:TESTING/tizonia/tizonia.spec?expand=1

Luigi already ported this over to meson, so that's all good.

[gdesmott]

According to @CapOM boost should only be needed when building the player but atm it's always required. Is that normal?

Also what is your roadmap regarding merge and release of the develop branch? I'd rather use a fixed release in our CI so we don't use a moving target.

[lgbaldoni]

@tizonia the meson version is here https://build.opensuse.org/package/show/home:alois/tizonia

This exception was included so that the player could be built in OpenSUSE (for some reason libspotify was/is not available there):

OBS doesn't take kindly to proprietary software. Especially when it's also unmaintained :slightly_frowning_face:

@gdesmott https://github.com/tizonia/tizonia-openmax-il/pull/674

I would also like to know if forcing a meson >= 0.53 requirement would be a problem for most.

[gdesmott]

I would also like to know if forcing a meson >= 0.53 requirement would be a problem for most.

It depends the kind of users and system you are targeting. Gstreamer depends on >= 0.48.0 but the Rust plugins needs >= 0.52. We don't rely on packaged meson for our CI so we can easily upgrade if needed.

[tizonia]

@gdesmott

boost atm it's always required. Is that normal?

Good point. The minimal build does not require boost libs, just some header-only dependencies in the OMX IL Resource Manager. Actually, these could easily be replaced, to make the minimal build completely boost-free. https://github.com/tizonia/tizonia-openmax-il/blob/088b4ec7d07591c10cde2818d0961c44527fe980/rm/tizrmd/configure.ac#L84

I'll take care of these changes.

roadmap regarding merge and release of the develop branch?

The develop branch is going to be merged to master very soon for 0.20.0. It's just a matter of finishing the hardening of a couple of features there and some documentation. This could take another 1-2 weeks. At that point, the plan is to keep Autotools as the 'stable' build system and Meson in 'preview mode' for another release cycle. This is required as we need some time to gain experience before autotools can be completely replaced. E.g. debian packaging still works with autotools; it will need to be migrated in the next cycle. So it is worth keeping in mind that Meson may need some tweaking in this period. We'll try and make an effort to not break you if you decide to use it from now.

@lgbaldoni

the meson version is here https://build.opensuse.org/package/show/home:alois/tizonia

Sorry, I missed your repository!. It is true, libspotify is non-free and unmaintained (hopefully Spotify will have a hard time breaking it in the near future due to all the legacy integrations out there).

like to know if forcing a meson >= 0.53 requirement would be a problem for most

I thought about this and I believe most likely it won't be a problem (meson is only a pip install away after all). Hopefully, the next cycle should help in deciding this type of thing.

[tizonia]

boost atm it's always required. Is that normal?

Luigi's PR #674 is now merged.

[gdesmott]

boost atm it's always required. Is that normal?

Luigi's PR #674 is now merged.

It's still looking for boost headers when building: https://gitlab.freedesktop.org/gdesmott/gst-ci/-/jobs/1517457

[tizonia]

Sorry, that should be fixed now: 15aec5b9b7b55f9195e9b97d155cf31e8d34e266

[gdesmott]

Thanks; that works now.

[gdesmott]

Just to be sure, we do still need log4c in minimal builds right?

[tizonia]

Yes, that's required.

[gdesmott]

It's all good now for our CI so I think we can close this bug.

Do you have any plan to make a release with the meson support?

[tizonia]

Hi Guillaume, yes, definitely. Still in the process of tidying up things in the 'develop' branch. The new release should hopefully be out towards the end of next week.

[tizonia]

@CapOM / @gdesmott

Please note that develop has just been merged to master.

[gdesmott]

Thanks, I submit a MR to add it to GStreamer's CI.

[tizonia]

Hi @gdesmott

I wanted to mention that we have started the deprecation of the internal libev (https://github.com/tizonia/tizonia-openmax-il/tree/master/libtizplatform/src/ev). Both meson and autotools now give preference to the external library if it is found.

In the next cycle, I will remove the internal one. To avoid breakages in your CI, please make sure that the development version of libev (e.g. libev-dev) is installed on your CI server.

Thanks!