Drop WebRTC in favour of QUIC-streaming

[wanjohiryan]

Low latency video and audio streaming from server to client with Webtransport

[ehfd]

Wow, this project is going on really fast. WebTransport only months after the RFC. 👍 👍 👍

[wanjohiryan]

Hi Kim @ehfd ,

Thanks. It's an honour finally meeting you. (Your projects have been an incredible inspiration.)

Yeah, we will be using WebTransport. It is much faster than WebRTC and Websockets, and is scalable too.(You expose one udp port per container like selkies-gstreamer)

[ehfd]

@wanjohiryan

Hi! Your development traction is very quick, and I believe that is rewarded by your massive accumulation of stars. Please continue the great work, and provided since your project has massive traction, please do reach out to me if you have anything that needs my help, I will happily assist you (https://discord.gg/wDNGDeSW5F).

We are also trying to make multiple connections work per web app, so if it works, please do consider taking a look at selkies-gstreamer later on too!

[ehfd]

Hi! @wanjohiryan

https://github.com/selkies-project/docker-nvidia-egl-desktop https://github.com/selkies-project/docker-nvidia-glx-desktop

There are two new releases, which allows recent nvidia/cuda images instead of the not-updated nvidia/cudagl images (that you are using now in gamevm/base/Dockerfile). License also has been relaxed to MPLv2 instead of GPLv3, both are compatible with your license.

Other than that, we now use the wider used KDE Plasma instead of Xfce/MATE.

[wanjohiryan]

Oh, hi Kim @ehfd

There are two new releases, which allows recent nvidia/cuda images instead of the not-updated nvidia/cudagl

That's good news. I will definitely check them out. Thank you so much.

I also have a question, how do you enable gamepad/joysticks in docker without passing in /dev/input from the host? Do you mind sharing your 'trick'?

[ehfd]

I also have a question, how do you enable gamepad/joysticks in docker without passing in /dev/input from the host? Do you mind sharing your 'trick'?

Unfortunately, you can't. Only way to do it without privileges is to use the X11 input interface. While I believe that there is technically a way to make it possible (check KasmVNC and CloudRetro who sorta did them), it remains to be tested.

My proposal is a "game mode cursor" where the cursor's absolute position always stays at the center, and the cursor moves relatively to the direction of the joystick/gamepad during the movement within the device (capturing this movement through the HTML5 Gamepad API). This would also work out with mouses on games that go weird on absolute cursors. We will attempt this.

One other way is to create a device for the X11 Joystick extension within the container somehow.

Active discussion in: https://github.com/selkies-project/selkies-gstreamer/issues/55 https://github.com/selkies-project/selkies-gstreamer/issues/28

[wanjohiryan]

Unfortunately, you can't. Only way to do it without privileges is to use the X11 input interface

Oh, that's sad tbh. I have looked at Kasm from a DiD point of view (from here) and it requires /dev/input from the host.

Problem : I want qwantify to feel native. This is by integrating gamepad vibrations, LEDs etc, which is not possible if i map HTML5 Gpad API to mouse and keyboard events on the server.

Solution1 : I looked at cloud-morph (i have a fork of it myself from a while back). From here, i could create a prebuilt wineprefix, which would come preinstalled with vigembus (plus other libraries) and then using the vigem-client on the server with c# or python, i could create gamepads, mouse and keyboard events.

Solution2 : With /sbin/modprobe uinput, i was thinking of creating the /dev/input directory in the container manually. I had tried this once before, with one of your docker images (nvidia-glx)[https://github.com/ehfd/docker-nvidia-glx-desktop] and it worked(to some extent :). However, i kept running into errors with neko.

Solution1 is unscalable but doable, hence i might take that route. However, before i start going down that rabbit hole. what advice would you give me, from your experience?

[ehfd]

Oh, that's sad tbh. I have looked at Kasm from a DiD point of view (from here) and it requires /dev/input from the host.

This linuxserver container is not what I advise. Kasm is unrelated to /dev/input in their game cursor mode. Try installing and taking a look at the game cursor function of https://github.com/kasmtech/KasmVNC. This works by sticking the cursor always to the center of the screen and only passing relative movements instead of sending the absolute position. This way no /dev/uinput or the kernel module is involved.

Solution1 : I looked at cloud-morph (i have a fork of it myself from a while back). From here, i could create a prebuilt wineprefix, which would come preinstalled with vigembus (plus other libraries) and then using the vigem-client on the server with c# or python, i could create gamepads, mouse and keyboard events.

We actually have a similar solution in https://github.com/selkies-project/selkies-gstreamer/issues/28#issuecomment-1354568918.

Solution2 : With /sbin/modprobe uinput, i was thinking of creating the /dev/input directory in the container manually. I had tried this once before, with one of your docker images (nvidia-glx)[https://github.com/ehfd/docker-nvidia-glx-desktop] and it worked(to some extent :). However, i kept running into errors with neko.

Does this ultimately work without modprobing the kernel module in the host? If it does, this is actually the preferable method. But if not, NOT usable on unprivileged docker (and adding the required privileges are unsafe and arbitrary users can take advantage).

[ehfd]

I am currently a bit busy, but I should be able to come back with this in some time.

[wanjohiryan]

This linuxserver container is not what I advise. Kasm is unrelated to /dev/input in their game cursor mode

Oh sorry, i thought they were related. My bad.

Does this ultimately work without modprobing the kernel module in the host?

Yes, you modprobe the running container only . which creates the /dev/input directory for virtual gamepads.

I am currently a bit busy, but I should be able to come back with this in some time.

Thank you for your time. I hope to hear from you soon.

Also, am kind of looking for a cofounder (qwantify is growing way too fast, and i can't seem to keep up and i would really need someone like you). If you're interested, we should definitely jump on a call sometime.

[ehfd]

Yes, you modprobe the running container only . which creates the /dev/input directory for virtual gamepads.

I should try this soon and see if it works as how I think. Thanks for this information.

Also, am kind of looking for a cofounder (qwantify is growing way too fast, and i can't seem to keep up and i would really need someone like you). If you're interested, we should definitely jump on a call sometime.

I do this in my part-time and I have lots more things I do at a timeframe, but I would like to help out in ways I could. I can also link you to colleagues who could actually do development as well. I'm available at https://discord.gg/wDNGDeSW5F.

[wanjohiryan]

Marking this as stale, as it being superceded by the new pr #36

Thanks @ehfd i will definitiley keep in touch. If you ever need me, you know where to find me.

Happy hacking :)