Video Players on Linux Native

Describe the bug?

Video players generally do not function correctly on the Native Linux build of Resonite. Non-Youtube videos (.ogv, .vp8, and .webm) will play through the Unity Native video player, but Youtube videos fail to play when using libVLC.

Proposed Solution

There are three different issues that need to be addressed:

Incorrect Plugin Directory: The UMP plugin libraries are stored in the incorrect directory. They are currently in Resonite/Resonite_Data/Plugins, but they should be in Resonite/Resonite_Data/Plugins/x86_64.
Missing yt-dlp Binary: The yt-dlp binary is not included in the Linux Native build.
Incorrect yt-dlp Initialization: The code that initializes the yt-dlp binary doesn’t work properly on Linux Native. This code is contained in the VideoTextureProvider component. For Windows, it correctly finds the game’s AppPath and appends "RuntimeData\yt-dlp.exe" to it, which correctly finds the yt-dlp binary. However, on the Linux version, it only searches for the string "youtube-dl". This should be adjusted to find the correct binary.
```
// VideoTextureProvider.cs
private async Task<bool> LoadFromVideoServiceIntern(Uri url, CancellationToken cancellationToken)
{
if (youtubeDL == null)
{
    if (base.Engine.Platform == Platform.Linux)
    {
        youtubeDL = new YoutubeDL("youtube-dl");
    }
}
}
```

To Reproduce

Attempt to play a Youtube video using a Resonite video player configured with libVLC.
Observe that the video fails to play or exhibits playback issues.

Expected behavior

Videos should play back using the selected engine, specifically libVLC.

Screenshots

No response

Resonite Version Number

Beta 2024.6.11.74

What Platforms does this occur on?

Linux

What headset if any do you use?

No response

Log Files

frozenreflex - 2024.6.11.74 - 2024-06-21 18_14_09.log

Additional Context

No response

Reporters

@xlinka @frozenreflex

After (patchwork) fixing these issues on my own system, YouTube playback still does not work out of the box.

From my testing, the main issue seems to be that the only streams Resonite decides to play are ones with video and audio combined. The only thing Youtube serves with video and audio in one container is (iirc) h264 in mp4. Resonite on Linux native refuses to play back these formats (even with working UMP and libvlc). More open formats like vp9 and opus are served by youtube, but are in separate containers (although Resonite can play these separately out of the box, it chooses to use the combined audio+video format from youtube-dl).

Resonite can play these more open formats, but only separately, since it can only play one stream with one video player. By modifying youtube-dl output, it is possible to set up such a construction with 2 video players in game. To play these formats in a single video player, they need to have their streams combined into one container before reaching Resonite.

I have created a bash script which can be inserted into Resonite/youtube-dl, which acts as a shim between yt-dlp on the system, and Resonite. It downloads the video and audio streams separately, and with ffmpeg installed, combines them into one container. Then it hosts a webserver, and responds back to Resonite with what yt-dlp would've responded with, except only a single format, which points to a localhost URL. This script (+ dependencies) allows Resonite to play back youtube videos with any regular in-game video player.

youtube-dl.gz (gzip compressed, since github does not accept uploading arbitrary files, for some reason, but gzip is fine)

There is yet another issue after fixing the executable path for youtube-dl. Any livestreams crash Resonite. The streams themselves aren't a problem, but the metadata itself somehow causes a crash. I confirmed this through the use of the script, it should be simple to modify the script to ensure it is the metadata of livestreams, not the video streams themselves.

Playback of h264 is a relatively large issue to fix, especially considering how each Linux system behaves slightly differently in that regard (GPU, drivers, distro, installed packages, etc). Alongside that, the licensing of the format usually doesn't go well with applications on Linux wanting to decode video. I'm not sure if this issue will be fixed before youtube decides to block watching videos without an account, which would break youtube playback in all versions of Resonite.

Yellow-Dog-Man / Resonite-Issues