Support for clapper gstreamer based media player

[josch]

Hi,

some players support being controlled via the MPRIS dbus protocol:

https://specifications.freedesktop.org/mpris-spec/latest/Player_Interface.html

Is the functionality offered by this protocol enough to allow being controlled by syncplay? Is there a minimal example of what is needed to add a new player to syncplay/players?

[Et0h]

Syncplay has excellent compatibility with mpv that could not be replicated using the MPRIS dbus protocol or indeed using any of the current APIs for major media players.

For 99% of PC use cases the best media player to use with Syncplay is mpv or an mpv-based media players like mpv.net. We already support a number of other players as a backup, and I don't see any strong grounds to justify the additional development and maintenance burden of supporting additional media players (or to change the API used to control currently supported media players).

However if there was a significant use case which you had the time and energy to be the lead developer/maintainer on then I'd be happy for you to let me know and I will go into the details of minimum viable implementations.

[josch]

I agree, syncplay is amazing with with mpv. The problem is, that mpv cannot do hardware decoding of video on platforms that are too slow to do it in software. My current system has 4x ARM Cortex-A53 cores at 1.5 GHz (NXP/Freescale i.MX8MQ). These are too slow to do play anything more than 480p without dropping frames. The good news is, that the platform also comes with a hardware decoder (Hantro G1 and G2) which, since Linux 6.1, can also do 1080p@60fps h264/avc1 and h265/hevc decoding in hardware. The problem is, that ffmpeg based players like mpv do not support this hardware decoder. There exists a patch stack against ffmpeg to add support for v4l2 requests that have now been rebased again and again for years but nothing is getting merged into ffmpeg proper. As a result, projects that rely on hardware decoding to work have carried these patches as well. Here an example for Libre ELEC, a distribution to run Kodi media center even on platforms like the raspberry pi (which also has a processor that is too weak and needs support of a hardware video decoder): https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch

But I'm not using Libre ELEC but a normal Linux Distro (Debian) and there, ffmpeg is not getting patched. My only way to watch 1080p video is to use a player that is able to utilize the hardware decoder of my platform. One of the players that can do it is clapper which is using gstreamer as its backend. And to remote control clapper, MPRIS can be used: https://github.com/Rafostar/clapper

I'm currently evaluating my options. I don't know how hard it would be to add yet another player backend to syncplay. I do not need it to be able to do much. Basic support (hitting play and pause at the right times) would be sufficient for me. But I don't know if that is enough.

[Et0h]

Thank you for explaining your use case.

I've now written a quick guide at https://github.com/Syncplay/syncplay/wiki/Guide-to-adding-support-for-a-new-media-player-to-Syncplay which should give some pointers on how to add support for a new media player.

You can't just add support for a general protocol, you need to support a specific player with specific known paths and path patterns, but if a different player uses the same protocol then support can be expanded either by modifying that player's implementation (if the implementation is identical) or by making a variant based on the base media player implementation.

In terms of https://specifications.freedesktop.org/mpris-spec/latest/Player_Interface.html you would need to implement the following methods:

• Play and Pause, linked to Syncplay's setPaused method.
• Seek, linked to Syncplay's setPosition method.
• OpenUri, linked to Syncplay's openFile method.
• Quit, linked to Syncplay's drop method.

You would also need to implement the following properties:

• PlaybackState and Position, linked to Syncplay's askForStatus/updatePlayerStatus methods.
• The xesam:url and mpris:length metadata, linked to self._client.updateFile which should be run through a thread. See mpv/VLC implementations.

Setting Playback_Rate is optional. You can set speedSupported to False if you don't want to implement this.

You might also need CanPause, etc. if necessary to queue requests for when the player is ready for the command. You might need some trial and error to figure this out.

[daniel-123]

Just adding a little bit of context, I've checked out one third party MPRIS library for mpv, and apparently it only implements select parts of the interface. It omits some that would be crucial for Syncplay to work at all.

This just doubles down on what @Et0h said - it really needs to be on player-by-player basis. From cursory search of support by various pieces of software, it seems to be largely undocumented. Lots of work would be needed to test various players and given that it's seemingly relatively niche interface - possible breakage with updates.

[Et0h]

I've not tried it out, but on the face of it https://github.com/Rafostar/clapper/blob/d0304d4dedd3b58fdf498f5979aa1ad538ff6ec9/lib/gst/clapper/gstclapper-mpris.c seems like a sufficiently full implementation of MPRIS to allow for Clapper to be supported by Syncplay. I'm not sure if it supports PropertiesChanged signal though, meaning you may need to regularly poll to see if the Uri has changed (and after a file has loaded to see if the length has changed).

Presumably you would need to connect to the D-Bus protocol using one of the libraries listed at https://wiki.python.org/moin/DbusExamples - it is possible that https://pypi.org/project/txdbus/ is the best option as it is Twisted based and Syncplay uses Twisted, but I've not confirmed this. If it works QtDBus might also be good if it can be made to with Syncplay's PySide implementation of PyQt but that might cause problems for someone who uses Syncplay in console mode because their system doesn't have pyside.

The main deficiency of MPRIS seems to be the lack of OSD support. This means you have to basically just ignore all the messages and people will need to keep an eye on the Syncplay window for messages. It also doesn't allow for chat messages to be entered from within the player like mpv can do.

[josch]

I've now written a quick guide at https://github.com/Syncplay/syncplay/wiki/Guide-to-adding-support-for-a-new-media-player-to-Syncplay which should give some pointers on how to add support for a new media player.

That guide is super helpful, thank you! I have the following points of feedback:

• mention that run() needs to call client.initPlayer with the new player instance and not just return that instance
• it seems that UiManager.showOSDMessage calls self._client._player.displayMessage without checking whether the client supports OSD messages
• document the time format expected by updatePlayerStatus() and setPosition() -- is it floating point seconds?
• is openFile() supposed to immediately reset the position of the video to 0 and pause it?

I have a proof of concept locally and now just need to find a second computer to test this out. :)

[Et0h]

Thanks for the feedback. I've updated the documentation to reflect the clarifications below.

• mention that run() needs to call client.initPlayer with the new player instance and not just return that instance

Done.

• it seems that UiManager.showOSDMessage calls self._client._player.displayMessage without checking whether the client supports OSD messages

Syncplay expects an implementation for this so a pass stub is needed even if there is no support for OSD messages in the media player. I think generally speaking all methods need to be implemented even if it just for a pass.

• document the time format expected by updatePlayerStatus() and setPosition() -- is it floating point seconds?

I've not dealt with this part of code in a while, but according to the basePlayer.py this uses floating point seconds.

• is openFile() supposed to immediately reset the position of the video to 0 and pause it?

It depends on the circumstances.

Reset position: The Boolean resetPosition value is used to specify whether Syncplay wants the player to set position to 0 when the file is loaded or to retain the current (previous) playback position. If resetPosition is True then position should be reset to 0, and otherwise it should be the previous position (which might require the position to be set to self._client.getGlobalPosition())

Pausing: I've not worked on that part of the code for a while, but in my understanding openFile is not supposed to change the paused state from what it was before the openFile (so you may need to set the state to (which might require the paused state to be set to self._client.getGlobalPaused()) but in some cases Syncplay will try to pause at the same time as the playlist advances if not everyone is ready. This is accomplished through a setPaused command being sent around the same time as the openFile command.

I have a proof of concept locally and now just need to find a second computer to test this out. :)

I hope it works! You might be able to test with two instances on one PC if you use a lower-enough resolution video.

[josch]

Thank you for your quick reply! :)

I hope it works! You might be able to test with two instances on one PC if you use a lower-enough resolution video.

The biggest remaining blocker right now is, that I'm stuck with what dbus library to use. If I use the dbus-python binding from freedesktop.org then it will work fine as long as I avoid listening to any signals. But it seems that the askForStatus method is getting polled several times a second (could this also be documented?) and thus it would not be good to do dbus calls every time that askForStatus is called but instead listening to PropertiesChanged signals and update local variables accordingly and then pass those to updatePlayerStatus in askForStatus. But listening to signals with dbus-python requires a glib or qt main loop it seems. I do not know whether somehow twisted can work together with dbus-python? I didn't find how to marry the two online.

So maybe using dbus-python is a bad idea and I should indeed look into a library like txdbus or QtDBus. I'd appreciate any recommendations on that front because it ultimately will impact the dependencies of syncplay for the clapper backend.

Thanks again!

[Rafostar]

I'd appreciate any recommendations on that front because it ultimately will impact the dependencies of syncplay for the clapper backend.

This is early WIP (lot of signals/features missing) and undocumented (due to being still unstable API), but if you are targeting Clapper specifically, one another option here would be using Clapper built-in WebSocket communication instead of MPRIS. The main benefits of it would be:

• already allows you to control player from different machine over network
• two-way communication in real time
• can be freely expanded in player (not limited by MPRIS spec)
• not Linux only feature

For a quick test, you can enable the "Control player remotely" option in Clapper preferences. Open some WebSocket test website like this one: http://livepersoninc.github.io/ws-test-page/ (I picked this as its open-source github pages hosted). Connect to: ws://127.0.0.1:6446/websocket (use same port set in Clapper prefs). Should connect. To play some video, send message with text: {"action":"set_playlist","value":["file:///path/to/video.mp4"]}. You can toggle play/pause:{"action":"toggle_play"} or seek {"action":"seek","value":60000000000} (time in nanoseconds).

Right now it only sends current playback state (paused/playing) back to the client in WS messages, but in the future once expanded and more stable I will add progress reporting, video title, etc. If this gonna become an often requested feature, might also consider adding "chat" thingy too.

[josch]

Hi @Rafostar how did you discover this issue?? :smile:

I think that's a good idea. Since MPRIS cannot do all the fancy things that syncplay supports and since syncplay support is by-player anyways, using a custom IPC protocol instead of MPRIS sounds like a great idea.

I'll file issues against clapper for missing features of the WebSocket API.

I'll rename this issue from MPRIS to Clapper support. Can we leave it open until I'm done implementing it? Thanks!