Current state of video playback?

[morphogencc]

I was wondering what the state of video playback in Cinder was these days, and what the roadmap for future additions might look like.

Quicktime was deprecated in Windows in 2016, and OSX in 2017, but still seems to be the core video player included with Cinder. It seems like there's a variety of community-supported blocks out in the wild (this post in the forums does a good job of reviewing them), but none of them seem to have made it back into the core library yet; many of the blocks also seem to be either abandoned or not supported anymore.

I see that there was some discussion about using gstreamer as a cross-platform video solution, but it looks like it still requires some manual configuration with CMake (and downloading gstreamer separately). I think it'd greatly simplify a lot of users' lives if there was a universal, cross-platform video playback solution that just worked "out-of-the-box".

It seems like gstreamer was the solution at one point in the past, but it's not clear from checking past issues why it wasn't fully integrated -- looks like work stopped around 2018. Are there any updates on it?

If another library is necessary, libvlc or ffmpeg could be other candidates for cross-platform video. libvlc would probably be the easiest for 80% of use cases, but isn't quite as fully featured as ffmpeg (and doesn't feature hardware acceleration).

This is currently a pain point for my own work, so I'm happy to put in the time to get the video modules in working order, but I do want to make sure that I have a good understanding of whether the community's work stands right now!

[richardeakin]

I would say that as of now, the best bet for video support is using a third-party addon / cinderblock, since none of the options available have permissable licenses for including in cinder core itself, or require large binary dependencies such as gstreamer. We had hoped to use Media Foundation on windows and all attempts ended with serious reliability issues to the point that the efforts towards core integration have been abandoned (however Cinder-WMFPlayer is still widely used).

ffmpeg is an obvious candidate and although I don't have links off the top of my head I know a few people in the community have made use of it in cinder projects. Issue with integrating it into core is again, licensing (LGPL). But if this is okay with your project, it makes a great candidate to be utilized as an add-on.

There have also been a handful of implementations I've seen that use NVidia APIs directly for reading compressed textures on the gpu, along with a H264 parser, however none that I know are public. That sort of thing takes a ton of work.

[morphogencc]

@richardeakin Thanks for the thorough response -- that all makes sense! I got Cinder-WMFPlayer working for my own short-term purposes, but I think I may work on using the latest version of FFMPEG for my ongoing work. I was hoping that perhaps it could be an opportunity to contribute to the core library, but it sounds like licensing makes that a pain; c'est la vie!

[richardeakin]

Yea that's the gist of it, though a solid working ffmpeg cinderblock would help the community immensely. Especially if it can take advantage of the gpu decoding path.

[axjxwright]

Just to throw an alternative in the mix, I have released a block for video playback on windows here. It's only very new so hasn't been battle hardened yet but I am using it in production and it's been pretty stable so far.