Controller input and refresh rate feels off on Quest 2

[Espionage724]

I'm not sure if Relive VR is enforcing some sort of limit (like 72Hz), but controller movement and overall refresh rate just feels off. I enabled 90Hz via the adb command.

Using Virtual Desktop on the same computer presents a more immersive experience.

I don't know how to ideally check latency or refresh rate, but I'm using Beat Saber as my main test.

[GennadiyAMD]

@Espionage724 - Could you please attach the c:\Program Files (x86)\Steam\logs\AMDWirelessVR.log file? Thanks!

[Espionage724]

Sure: AMDWirelessVR.log

Although looking at that log, it looks a little small; I was doing some testing earlier and turned Relive VR off; would that have wiped out my previous logs? Should I go try to play some games and upload the log again?

[GennadiyAMD]

@Espionage724 - it looks small because you just started SteamVR, but didn't actually start a VR session :) Sorry, I should've made this clear from the beginning - my fault.

1. Start SteamVR
2. Start the ReLive VR app on the headset
3. Play for a few (2-3) minutes
4. Send the log

Thanks!

[Espionage724]

AMDWirelessVR.log

With Beat Saber, I noticed that the audio that comes out of the headset itself seems delayed compared to what comes out of my computer speakers. If I mute the headset audio and play with the computer speakers, Beat Saber feels a lot better to play, but something still seems odd with how the game looks; like the framerate could be smoother.

[GennadiyAMD]

@Espionage724 - thanks. The log shows that you're streaming at 72fps. This might account for what you're describing as "looking odd" - when the frame rate of the display is different, you could see some jerkiness that would be hard to describe. Will look into why this is happening - need to investigate.

Sound being delayed a little compared to speakers is normal. We have to buffer at least a little, otherwise, because network latency is not constant, sound would break up when the next packet doesn't arrive in time. Not much can be done about this, unfortunately. This would be a common problem with any streaming solution.

[eywhat]

Can the sound also be exhibited completely? Would this improve the latency somewhat?

[GennadiyAMD]

@eywhat - you mean not streaming sound to the headset and go with something like Bluetooth headphones? This would not have much effect on latency. The network itself is actually not the largest contributor to the overall latency, the network latency is usually in single digits ms with a good connection. Most of the latency comes from encoding, decoding, rendering and the fact that display refresh on the headset cannot be synchronized with rendering on the PC - each runs in its own time domain. The network is only about 10% of all this. Besides, the video is encoded to 50-100Mbps, while sound is 256-320kbps - that's less than 1% of the data being streamed to the headset...

Going Bluetooth - if you have BT headphones, you can certainly try, just turn the headset volume down for the sake of experiment. But results would vary depending on the headset as BT suffers from the same kind of interference as WiFi and buffering is the only remedy available. So BT would also introduce some lag to sound, the question is whether the headphones buffer more or less than ReLive VR. Wired headphones is the only thing that would not delay sound, but they defeat the purpose of wireless streaming.

[Espionage724]

Sound being delayed a little compared to speakers is normal. We have to buffer at least a little, otherwise, because network latency is not constant, sound would break up when the next packet doesn't arrive in time. Not much can be done about this, unfortunately. This would be a common problem with any streaming solution.

I'm not sure how Virtual Desktop is doing it, but I don't notice any sound latency with their streaming solution. Specifically, it seems the audio on the headset is in-sync with the video with VD, whereas the audio is a bit delayed with Relive VR.

[GennadiyAMD]

@Espionage724 - how much of an delay are you hearing? I know you can't really measure it, but an estimate? Is it more like 0.1s or 0.5s? And if you compare the sound in VD to speakers connected to the PC, is there a delay there?

[GennadiyAMD]

@Espionage724 , @eywhat - version 1.0.25 of the client adds support for 90Hz on Quest2 - please try this update. I'm going to close this issue for now, if you experience any issues with 1.0.25, please feel free to re-open or file a new one.

[Espionage724]

@Espionage724 - how much of an delay are you hearing? I know you can't really measure it, but an estimate? Is it more like 0.1s or 0.5s? And if you compare the sound in VD to speakers connected to the PC, is there a delay there?

This still happens with 1.0.25 Relive VR app and 20.12.1 graphics drivers. The delay seems to be about 0.25-0.5 seconds.

[GennadiyAMD]

@Espionage724 - 0.25 sec is normal. There's a little bit of buffering needed, or sound would break up frequently - you can't repeat an audio sample when the next one doesn't arrive on time, unlike video where you can present the last frame - with sound it would be very noticeable. I can hear some delay in VD as well when I play with the speakers in the monitor on - can't easily measure it, but there's definitely some.

[paranoidandy]

I'm not having this problem, but just wanted to comment on ways of determining audio and video latency "fairly" accurately. Some rhythm games like Audica from Harmonix have separate calibration settings for video delay and audio delay and can give you really good information. I would say 0.25s is very very laggy. VD users aim for 35-40ms of latency as a "perfect" setup, and see up to about 60ms total latency as pretty acceptable.

Personally, when running the Oculus PC version of Audica (through ReVive, in SteamVR, not using the VD stand-in for the Oculus runtime) I get 45ms latency for both video and input with VD, but I get 100-120ms latency with ReLive VR. I've run this several times and the VD latency is always pretty consistent, but the ReLive VR results vary between 100 and 120. I'd like to take ReVive out of the pipeline but I only have the Oculus version of Audica to test.

[GennadiyAMD]

@paranoidandy - when you're quoting latency numbers, which latency are you talking about - video or audio? And where do you take the numbers from when you say that ReLive gives you 100-120ms? You can see our numbers in the AMDWirelessVR.log in Steam\logs, although I'd refrain from direct comparison with VD as we don't know how VD measures it, so it might not be an apples-to-apples comparison. And from the logs you sent earlier, you're getting less than 50ms (quote from the second log you sent):

2020-10-16 21:59:26.382 22F0 [NetworkServer] Info: Average latency: full 44.59(average 45.32) client 17.82 server 23.99 encoder 9.74 network 2.78 decoder 7.18, frame time 13.89 ms 2020-10-16 21:59:29.590 22F0 [NetworkServer] Info: Average latency: full 44.22(average 45.31) client 17.51 server 24.01 encoder 9.77 network 2.71 decoder 7.15, frame time 13.89 ms 2020-10-16 21:59:32.798 22F0 [NetworkServer] Info: Average latency: full 44.86(average 45.31) client 17.94 server 24.11 encoder 9.78 network 2.81 decoder 7.33, frame time 13.89 ms 2020-10-16 21:59:36.005 22F0 [NetworkServer] Info: Average latency: full 45.16(average 45.31) client 18.05 server 24.14 encoder 9.77 network 2.97 decoder 7.34, frame time 13.89 ms

And that was at 72fps. Now that you're running at 90fps you should see them even lower by a couple of ms. Again, since this includes rendering, you have to run the same content the same way when comparing numbers. I'm not familiar with Audica, but Steam Home or something like Serious Sam are a pretty good test as they're lightweight enough in terms of graphics, so you're really seeing the latency the wireless pipeline adds.

You can't really measure latency on video alone or on audio as it starts on one device and ends on another device and the clocks on the sender and the receiver are unsynchronized. Audio is not played as the result of a motion. The latency on video and audio is not the same because of the buffering I mentioned earlier.

[paranoidandy]

I haven't posted any logs. Those were from someone else. I just joined the conversation because I'm interested in getting the best performance out of Radeon ReLive and latency is important. Sorry to be confusing.

I'm talking about perceived audio and visual latency when measured using an in-game delay calibration function. The game has a calibration tool that allows you to adjust the ms delay + or - until you see a metronome-style on screen element line up with an audio click. It's in theory designed for people with complex a/v systems or tv's with high processing that introduce delay. But it's a useful tool in this context.

The game is Audica by Harmonix.

[GennadiyAMD]

@paranoidandy - yes, my bad, the log came from somebody else, sorry about that. Could you then post yours? It's under c:\Program Files (x86)\Steam\logs\AMDWirelessVR.log. Please make sure you play something for a couple of minutes and then close SteamVR before capturing the log.

If there's a problem, let's figure it out.

[paranoidandy]

AMDWirelessVR.log

Here you go. I've looked inside and the numbers look pretty ok (~50ms) but the perceived delay in game was the same as I described before. I got 120ms delay on the video calibration, and 135ms delay on the input test.

I tried to take a video but then remembered Quest video recording has audio out of sync so it wouldn't be useful.

[GennadiyAMD]

@paranoidandy - I'm seeing fairly healthy numbers in your log ~49-50ms and this is measured as a round trip time from capturing a head pose to submitting a frame rendered with this pose for presentation on the headset. The only thing that this method cannot account for is the time from when the frame was submitted to when it actually appears on the screen - this happens in the hardware, so it could be half-a-frame short on average. At 72fps this would be around 7ms, 13.6ms worst case, 0ms best case. I'm not sure what you mean by "120ms on video calibration" - is that the delay between audio and video?

Not sure about what the "input test" means either. I've tried googling for it, but don't see anything meaningful, could you please elaborate more on what you (and the game) are doing? Thanks!

[paranoidandy]

Ok, I'm an idiot. I thought that there was an inconsistency between the audio and the visual which lead me to try this calibration thing, but actually I now think the issue is the variability in the latency as the gameplay changes.

And when I used the calibration tool, I had it a full beat off - so I was lining up the next beat to the "current" visual cue, if that makes any sense.

It wasn't that I thought I was getting 100ms latency, I've checked the logs before and knew I was getting sub-50ms, it's that I thought the audio was lagging behind. Anyway, I was wrong. Sorry to waste your time!

[GennadiyAMD]

@paranoidandy - no worries! No time is wasted if it leads to more knowledge, a better understanding and a happy user. It always works out this way :)

Yes, I understand what you're saying, sort of, as much as possible without actually seeing it. Network latency can be quite variable, and while on average it's typically within single-digit ms, you might get occasional spikes up to several hundred of milliseconds. You wouldn't notice it in general when you move your head because timewarp hides it, but it becomes clearly visible when you move controllers quickly enough. That's where you actually see the real latency and its variability. I suppose when you see a metronome that oscillates say, 2-3 times a second, and a single frame got delayed once by 200-300ms, you could easily misalign the video and the audio by one period and think that you're off by these 200-300ms.

At any rate, I'm glad you've figured it out. Do not hesitate to let me know if you find any problems - always happy to help!

[Espionage724]

needed, or sound would break up frequently - you can't repeat an audio sample when the next one doesn't arrive on time, unlike video where you can present the last frame - with sound it would be very noticeable. I can hear some delay in VD as well when I play with the speakers in the monitor on - can't easily measure it, but there's definitely some.

I'll have to see if there's some better measurement I can do, but I'm not really sure how to specifically explain the delay. With VD, if I use just the headset audio and play Beat Saber, it feels ok and no problem. With Relive VR, it feels off in some way, but if I use the computer audio instead, it feels better. It could be input delay instead also; controllers seemed to feel more floaty with the brief test I did.

Does Relive VR do anything to predict controller movement? If I heard right (haven't confirmed or asked about it), VD does some kind of predictive movement.

[GennadiyAMD]

@Espionage724 - yes we do prediction on both controllers and the headset. Don't know if we do it the same way as VD as I don't know how VD does it.

If you play audio on both the headset and the speakers at the same time, obviously they won't be completely in sync, but does the delay between the two sound more or less the same in ReLive and VD? I know, this would be very subjective, but still... This would, at least very subjectively, answer the question of whether it's in the sound or in controllers. The sound is obviously slightly delayed because of the buffering I mentioned earlier, but I'm curious if you perceive this delay differently?