Implement video player GUI component

[nightm4re94]

Many games play more or less important videos at some point, e.g. for in-game skill showcases, cut scenes, intros, and many more. So far, we have neither Resource management nor playback capabilities for video files. With pure java, this is not exactly a trivial thing to do, so we might need to use some external libraries for this.

Potential candidates

JavaFX (GPL)

It is possible to play back video in a JavaFX scene wrapped into a JPanel, but we need to consider this carefully, since JavaFX is not needed anywhere else in the engine right now and LITIengine is an AWT framework by design. Using JavaFX at one point would imply switching other internals such as rendering and UI to JavaFX as well, but that would entail completely changing LITIengine's scope...

Xuggler (LGPL-3.0)

While promising, Xuggler has been deprecated in favor of humble-video.

Humble-Video (AGPL-3.0)

The succesor to Xuggler, but Licensing may be incompatible.

vlcj (GPL 3.0)

A native vlc player instance to be embedded in Java applications.

Java Media Framework (JMF)

Truly an antiquity, the JMF may be a bit dated and lack support for many modern containers and formats. Still, it may be enough for our needs. example 1 example 2

Freedom for Media in Java (FMJ) (LGPL-3.0)

This library addresses some of the shortcomings of the JMF (e.g. codecs and containers), while retaining a fully JMF compatible API.

jcodec (BSD-2-Clause)

An implementation of the most common video and audio codecs.

GStreamer (LGPL-3.0)

Java bindings for GStreamer.

The requirements

• a video player GUI component to be used in ingame UI
• a cut scene screen, a.k.a. a full screen video player
• resource management for videos
• codec support for the most common codecs
  • H.264
  • MPEG-4
  • DivX
  • MPEG-2
  • HEVC (H.265)
• container support for the most common file containers
  • MP4
  • AVI
  • MOV
  • FLV
  • WMV

[Gamebuster19901]

Perhaps you should look at https://github.com/caprica/vlcj

Nevermind it's GPL

[Gamebuster19901]

Just a note:

VLCJ has an incompatible license (GPL).

OpenJFX however is compatible (GPL with Classpath Exception)

[Gamebuster19901]

JCodec actually looks very promising. I'll give it a shot.

It looks like the video player in jcodec's repository is unfinished. It doesn't exist on maven central and has some... interesting method names.

https://github.com/jcodec/jcodec/blob/4d90a3a79dcaf931b09145eff03a2ec174482a56/jplayer/src/main/java/org/jcodec/player/util/ThreadUtil.java#L36-L52

[nightm4re94]

Hahaha, what the hell😅

[nightm4re94]

So I reached out to the developers of vlcj to see if by any chance we could use their product despite incompatible licensing, here's what they said:

Hi,

I'm somewhat sympathetic to what you're asking, but I don't know how it could work.

I can't allow you to distribute vlcj to third parties in any way that would mean those third parties would not have to comply with my choice of license. If they're open source, then what's the problem with GPL - If they're commercial, let them pay a license fee. Seems perfectly reasonable to me.

It would need some proper legal advice to decide what, if anything, was possible here.

Regards,

-Mark

So while they are not entirely reluctant to let us use their product, they have legitimate concerns about the legal viability of such an endeavour. It would require a way of us redestributing vlcj while making sure that any derivations of our work would comply with GPL, and I honestly do not see such a way. Does anyone have any expertise on this field?

[Gamebuster19901]

It would require a way of us redestributing vlcj while making sure that any derivations of our work would comply with GPL, and I honestly do not see such a way. Does anyone have any expertise on this field?

GPL was intentionally designed to be incompatible with MIT, so it's not possible.

GStreamer seems like the best alternative. This might be helpful: https://github.com/gstreamer-java/gst1-java-examples/blob/master/src/main/java/org/freedesktop/gstreamer/examples/SimpleVideoComponent.java

[nightm4re94]

I thought so... Well if you still want to give it a shot, feel free to try GStreamer.

[Gamebuster19901]

I was able to get something sort of working with GStreamer, but there are many bugs with the GStreamer java bindings.

For example, you can't seek to a specific point in the video unless it is playing. And you have to wait ~120 milliseconds after playing before you can seek.

Anything more complex than pausing/playing the video doesn't really work at all.

My fork is here: https://github.com/Gamebuster19901/litiengine/tree/video

Also the fork is largely untested, there may be memory leaks etc.

[nightm4re94]

Great to hear you've managed to make some progress on this!

[nightm4re94]

As long as video support is not in the engine, I want to advertise using vlcj in your game (keep licensing in mind). Playing back a video in full screen with vlcj in LITIENGINE is as simple as these four lines:

    MediaPlayerFactory mediaPlayerFactory = new MediaPlayerFactory();
    EmbeddedMediaPlayer mediaPlayer = mediaPlayerFactory.mediaPlayers().newEmbeddedMediaPlayer();
    mediaPlayer.videoSurface().set(mediaPlayerFactory.videoSurfaces().newVideoSurface(Game.window().getRenderComponent()));
    mediaPlayer.media().play("path/to/video.mp4");

[AZ-0]

A bit late in the discussion, but I remember a similar issue had happened with Minecraft's server overlays, namely bukkit, craftbukkit, and spigot. The issue here is that while spigot is built on craftbukkit and craftbukkit on bukkit, bukkit is GPL but neither spigot or craftbukkit are (which caused legal issues to craftbukkit); but lo and behold! Spigot found a way to circumvent this issue: • The spigot API distributed to developpers doesn't expose craftbukkit (because doing so would entail the aforementioned legal issues) and instead provides a wrapper. • However, they also provide build tools, which compiles the spigot server (this one uses craftbukkit) from source using bukkit and craftbukkit open-source repositories; which mean they don't actually redistribute the GPL product and therefore don't need to be GPL themselve.

This is only a vague outline from what I remember and requires further investigation, but perhaps a similar approach could work in your case.

[Gamebuster19901]

perhaps a similar approach could work in your case.

Don't do this, just respect the wishes of the developer(s).

Currently, both GStreamer and JavaFX have working implementations and have compatible licenses so there are alternatives. You should only use vlcj if you are fine with licensing your code under the GPL.

[Gamebuster19901]

Perhaps we should take a look at https://github.com/bytedeco/javacv. It's licenced under apache so it's compatible.

It is a pretty big dependency though, and contains a lot of stuff litiengine doesn't need.

[Gamebuster19901]

This looks promising and has a compatible license: https://github.com/bramp/ffmpeg-cli-wrapper

I'll see if I can get something working soon.

[nightm4re94]

looks good indeed!