search

moonlight-stream / moonlight-embedded

Gamestream client for embedded systems
https://github.com/moonlight-stream/moonlight-embedded/wiki
GNU General Public License v3.0
1.48k stars 323 forks source link

v4l2 support on Raspberry Pi #809

Open c4tz opened 3 years ago

c4tz commented 3 years ago

Hey,

first of all: Thank you for providing us with this great piece of software!

I'm using moonlight-embedded on a Rpi2 running Arch Linux ARM and it has been working perfectly until this week. Unfortunately, the (new) kodi-rpi package requires this boot config, which seems to break moonlight.

When I revert the config back to only include

gpu_mem=256
initramfs initramfs.gz followkernel

moonlight-embedded works again, but Kodi does not.

There is no error in the verbose log (or anything different from when it is working at all), I can only hear a distorted version of the streamed audio and see nothing but the TTY I started it in.

I tried with both moonlight-embedded in version 2.4.11 (which does not install ATM, btw) I had before and a freshly installed moonlight-embedded-git.

Would it be possible to adapt to this config/drivers somehow?

cgutman commented 3 years ago

I suspect it's dtoverlay=vc4-kms-v3d that's causing the breakage. My understanding is that OMX and MMAL can't render to the display in true KMS mode (not fkms which is the current default on Pi 4). You have to render with the DRM APIs, which seem to only be used in the Rockchip video backend at the moment.

FFmpeg does have a V4L2 M2M implementation which can be used on the Pi (Moonlight Qt supports it). I haven't found the performance of V4L2 M2M to be great (using either EGL or DRM directly), but YMMV.

c4tz commented 3 years ago

Thank you for the elaborate answer @cgutman . However, I'm not quite sure what you're suggesting.

Do you mean I should try moonlight-qt (which I can't, because it is not supported on the Raspberry Pi 2)?

Is there a possibility (I did not see yet) to make moonlight-embedded use the V4L2 M2M codec?

Or do you mean I should find a way to use Kodi 19 without dtoverlay=vc4-kms-v3d?

cgutman commented 3 years ago

It should be possible to use the V4L2 M2M codec on the Pi 2, but I don't know if the performance will be very good. I'm surprised that Kodi is using KMS on the Pi 2. The KMS drivers are very immature on Pi 3 and earlier last time I checked, and it couldn't even render the Moonlight Qt UI correctly (see https://github.com/moonlight-stream/moonlight-qt/issues/555).

It might be possible to use MMAL directly to decode the frames then get a DMA-BUF back and pass it to drmModeAddFB2WithModifiers() to render with DRM directly. I found a similar approach (except it was a V4L2 request codec, rather than MMAL) to provide quite reasonable performance on the Pi 4 at least for the HEVC codec.

One thing you can try is changing dtoverlay=vc4-kms-v3d to dtoverlay=vc4-fkms-v3d. That may break Kodi but it should allow OMX/MMAL rendering to work in Moonlight.

c4tz commented 3 years ago

I compiled moonlight-qt-git on my RPI 2 in order to test V4L2 M2M with ffmpeg as you suggested, but unfortunately I only got about 1-3 FPS with it, so this path is probably not the one to follow on that device. The UI rendered correctly, though, in and outside of x11.

Afterwards, I tried setting dtoverlay=vc4-fkms-v3d, but it made Kodi buffer forever after one second of playtime. This tells me that in the worst case I'd just have to switch back and forth between including kodi.config.txt and gpu_mem=256 to either use Kodi or moonlight-embedded.

It might be possible to use MMAL directly to decode the frames then get a DMA-BUF back and pass it to drmModeAddFB2WithModifiers() to render with DRM directly. I found a similar approach (except it was a V4L2 request codec, rather than MMAL) to provide quite reasonable performance on the Pi 4 at least for the HEVC codec.

This seems like a good approach to me, but I'm not sure how I could test it, as I'm not (yet) proficient enough in C to dive into the code.