search

alvr-org / ALVR

Stream VR games from your PC to your headset via Wi-Fi
MIT License
5.52k stars 488 forks source link

VR driver rewrite / Linux support #269

Closed zmerp closed 3 years ago

zmerp commented 4 years ago

Add support for Vulkan and Linux. This should be the last step of the Rust rewrite. By the time this issue is reached, Valve could have added an OpenXR driver interface. In that case that should be preferred in place of OpenVR. Another option is implementing an OpenHMD driver.

Can someone provide a simple VR driver example for Linux? It doesn't need to be integrated in ALVR just yet.

EDIT: You can follow the development progress here: https://github.com/alvr-org/ALVR/wiki/Linux-Support-development-progress

frostworx commented 3 years ago

I gave ALVR a quick try with a Quest 2 these days on my Arch Linux system. The system should handle this easily (RX 6900 XT, AMD Ryzen 9 5900X, 32gb ram, 5ghz wifi as AP), setup is straightforward and works very fine, but SteamVR in the Quest hmd is slower than a slideshow. fps shown in the gui are fine though (somewhere around 55-60fps). Do you have a clue what the reason could be? I usually use my Valve Index, which of course works fine. Is it maybe possible that some Index configuration left-overs conflict with ALVR? I built the server component manually with bundled-ffmpeg, because system ffmpeg isn't compatible currently. Tried both nightly and latest stable apk without a difference, and nightly ubuntu server binary doesn't work. To be honest I haven't checked for any debug output or even logs, so maybe it is something obvious I simply missed.

Different topic, but configuration switch back to using the Index is not too seamless (might require a restart as else the index video is drawn in a window, vrpathreg.sh removedriver might be needed, and the ~/.steam/steam/steamapps/common/SteamVR/bin/linux64/vrcompositor symlink needs to be set back to the original vrcompositor) This is no bug-report or complain, but it might be a good idea to mention this somewhere. I'll test a bit more, but I thought it might be a good idea to already open an issue, maybe also to encourage others who test the linux build.

xytovl commented 3 years ago

Thanks for the feedback, are you running at full resolution? It might be the decoder in the headset that can't keep up with the high definition stream. We have not implemented foveated encoding yet, which would reduce the number of pixels to decode.

You are the first to come by with both a quest and an Index, on Linux. If you want to join the discord, there are sometimes discussions where we don't know if something is an ALVR bug or a SteamVR limitation (such as crash on amdvlk, corrupted menus in grid view).

For the symlink, it is supposed to not break the Index, but I guess there are many bugs as nobody could test.

ckiee commented 3 years ago

@frostworx

setup is straightforward and works very fine, but SteamVR in the Quest hmd is slower than a slideshow. fps shown in the gui are fine though (somewhere around 55-60fps). Do you have a clue what the reason could be?

Are you monitoring the FPS through the ALVR dashboard? (go to http://localhost:8082 in your browser if it doesn't open on it's own) Is it just the client FPS that is bad?

Is it maybe possible that some Index configuration left-overs conflict with ALVR?

I don't think so.

Different topic, but configuration switch back to using the Index is not too seamless (might require a restart as else the index video is drawn in a window, vrpathreg.sh removedriver might be needed, and the ~/.steam/steam/steamapps/common/SteamVR/bin/linux64/vrcompositor symlink needs to be set back to the original vrcompositor)

Let's keep track of this in https://github.com/alvr-org/ALVR/issues/665. (low priority though)

The system should handle this easily (RX 6900 XT, AMD Ryzen 9 5900X, 32gb ram, 5ghz wifi as AP)

Fancy specs but how close is your AP to the Quest? Is the PC connected via Ethernet? You can always try using ALVR wired.

We have a channel where we discuss this stuff in the Discord server.

frostworx commented 3 years ago

Hello, thank you very much both @xytovl and @ronthecookie for your friendly and quick help! I haven't changed the resolution, so I assume it is running at full resolution. Will test more soon and report back if I find out something useful. IIRC I also tried x265, but will test that again to be sure. amdvlk is not installed, only using mesa-21.1.0-1 here. No big deal with the vrcompositor symlink, only wanted to mention it somewhere. Thank you very much for even opening an extra issue for it :)

Haven't checked localhost:8082 yet and only checked the fps in the ALVR gui. Will check this as well and report back. I did not carefully look at the SteamVR window ony my desktop, but I'd say its fps were normal. I bought an awus1900 just for this setup, as the previously used wifi stick was using 2.4ghz and I thought that was the bottleneck. I use the awus directly as AP, so the Quest directly registers as client at the pc. The Quest is not allowed to connect to the internet, so it very likely doesn't have the newest firmware and can't reach anything it wants to autoconnect with. Using ALVR wired is a good idea , will try that as well. Thanks again for your friendly support. I'll report back, when I have something useful to tell :) I try to avoid discord when possible, but might join later.

frostworx commented 3 years ago

Hi again, just a quick update: http://localhost:8082 fps and everything else look fine. Depending on the used resolution it has up to 60fps on client and 60fps server. Zero packet loss. The view in the Quest kind of looks like if it is rendered twice: A smaller window showing the whole view and in the surrounding border another view, which kind of looks like the resolution it should have. The controllers can be particularly seen though also on the upper border and not only at the bottom one. Maybe that is the main problem, and it kind of renders everything twice? I also had the feeling that it worked a bit better today than on the last try, but the core problem remains. When showing the hmd view as desktop window it looks normal and runs almost smooth, so I guess as expected. Haven't tested wired mode yet.

frostworx commented 3 years ago

Just tested in wired mode and the problem persists. I guess this scrcpy screenshot explains better what it looks like than my attempt above: alvr-linux

ALVR settings when taking the scrot: Video:

Headset:

If there's anything else which could help, please let me know.

xytovl commented 3 years ago

Foveated encoding is not implemented on Linux (but decoding is on the headset, which shuffles around the image). You should leave it disabled for the moment.

I mentionned it because at 100% Quest 2 resolution your are probably hitting the limits of the hardware decoder, and foveated encoding is a technique used to reduce the size of the image.

frostworx commented 3 years ago

Thanks will try this. Can't remember that I have touched it though. Maybe the initial wizard set it, when choosing some preset option?

frostworx commented 3 years ago

Unfortunately, disabling it didn't fix it completely, but the resolution looks correct now. Input (moving controllers) lags 5-8 seconds and head tracking lags about 1 second. So when moving the head I see the not rendered part (black) until the video arrives a second later.

statistics from the last try:

Alle Packets:   144592 Packets
Packetrate: 3091 Packets/sek
Gesamter Packet-Verlust:    0 Packets
Packets verloren pro sekunde:   0 Packets/sek
Alle gesendeten:    201 MB
Gesendete Rate: 34.508368 Mbps
Gesamte Verzögerung:    4537.491 ms
Encodierungsverzögerung:    15.066 ms
Maximale Encodierungsverzögerung:   35.248 ms
Transportverzögerung:   2316.711 ms
Entschlüsselungsverzögerung:    14.214 ms
Fec Prozentsatz:    5 %
Fec Verlust gesamt: 0 Packets
Fec Verlust pro sekunde:    0 Packets/sek
Client FPS: 52 fps
Server FPS: 54 fps

If there's anything else which could help, please let me know.

ckiee commented 3 years ago

I try to avoid discord when possible, but might join later.

@frostworx I've been guessing this might be the case for some people; maybe we can get an IRC or Matrix bridge going if more people are comfortable with that. (@xytovl, yay or nay?)

xytovl commented 3 years ago

I don't care the protocol as long as we don't fragment discussions. If someone wants to to the setup it's fine.

m00nwtchr commented 3 years ago

@lmarianski That isn't supposed to work! We haven't worked on that at all yet and no one has gotten it running before you so I have some questions:

  • What audio server are you running? (PulseAudio, PipeWire, pure ALSA)
  • Did you perform any extra steps in your audio server to get it to work?

  • If you're on PulseAudio, can you run pactl info and pactl list and send the output?

    • pw-cli info all for PipeWire.
  • Did you patch any code?

@ronthecookie I'm running PipeWire as a replacement for both PulseAudio and JACK, on Arch Linux (pipewire, pipewire-pulse, and pipewire-jack). IIRC i didn't mess with the PW config at all, here's the output of pw-cli while i'm hearing VRChat audio from the headset: https://pastebin.com/Z40hLytR For the audio settings i have Stream game audio selected (enabling microphone streaming does cause a error on connection), and "jack" as the audio device ("pipewire" and "pulse" cause it to crash or connection error, the rest do nothing). And nope i haven't edited any code

EDIT: Selecting the alsa audio device corresponding to my headphones seems to work too, it just didn't appear on the list earlier for some reason

ckiee commented 3 years ago

IIRC i didn't mess with the PW config at all, here's the output of pw-cli while i'm hearing VRChat audio from the headset: https://pastebin.com/Z40hLytR

It expired, please use a pastebin that won't expire so quickly (or ideally, never) next time.

For the audio settings i have Stream game audio selected (enabling microphone streaming does cause a error on connection), and "jack" as the audio device ("pipewire" and "pulse" cause it to crash or connection error, the rest do nothing).

Okay, that makes a bit more sense - PipeWire has the JACK bridge and our audio library does support JACK, but it does not have it enabled?

Let's move to #673.

ckiee commented 3 years ago

Oops, forgot to @lmarianski. Sorry!

IlyaMZP commented 7 months ago

Update: it seems that the IVRDriverDirectModeComponent interface that ALVR uses is not supported on Linux (ValveSoftware/openvr#749) and we still don't know anything about the IVRVirtualDisplay interface (ValveSoftware/virtual_display#5)

Good news! SteamVR beta 2.5.1 seems to add IVRDriverDirectModeComponent support to Linux: https://store.steampowered.com/news/app/250820/view/4146199229186243608

Vixea commented 7 months ago

yep I'm already working on it