I usually get packet loss but my network is fine

[vrokolos]

Moonlight with 60mbit streaming works perfectly fine on my network but ALVR will get packet loss and artifacts all over the place some times.

Is there anything I can do to fix this?

[polygraphene]

I think it is caused by ALVR client implementation. Though I don't know how Moonlight (or GFE) avoid packet loss, we may be able to get rid of packet loss by implement the same method as Moonlight. We need more investigation.

[vrokolos]

After changing server priority to higher along with all other VR services packet loss got identical to moonlight so I'm closing this issue.

[polygraphene]

I did some experiment on moonlight.

I used clumsy (really cool tool!) to cause packet loss on video streaming of moonlight. I got a quite nice result on 0~5% of packet loss. I didn't find any distortion of video even when 5% loss. I think this is because of FEC (Forward Error Correction, https://github.com/moonlight-stream/moonlight-common-c/pull/24) of moonlight.

Furthermore, I got good result on 10~15% of packet loss. When there are too much packet loss to correct error, I experience a bit broken picture, but the broken picture will disappear immediately. I looked into the H.264 stream on that situation, and I have found that moonlight send IDR-Frame to recover picture after packet loss adaptively.

Therefore, we need to implement:

1. FEC
2. Adaptive insertion of IDR frames

to achieve same performance as moonlight.

I think FEC takes a lot of time to implement. I will try adaptive IDR frame first.

[vrokolos]

Great. I would reopen this because yes I've seen some lost packets around 0.00001% that really break the image. I also believe moonlight has generally a better quality stream on h264 on same bitrate although I might be wrong since seeing dark compressed stuff right in your face and in 3d make the artifacts a lot more obvious

[polygraphene]

I have implemented adaptive insertion. Please check it! https://github.com/polygraphene/ALVRClient/releases/tag/v2.1.4

Actually, packet loss is mainly caused by IDR frame. Bacause IDR frame has large size and causes temporarily high bitrate of streaming. I removed periodical IDR frame on v2.1.4 and only send P-frame after client connected. And then if packet loss happens, we insert IDR the first time.

This procedure provide more clear video and low packet loss rate because we got more regulated bitrate.

[vrokolos]

Awesome. Will check it out tonight!

[vrokolos]

No this doesn't work correctly. When there is packet loss, the artifacts stay for a longer time and packet loss happens at the same rate

[vrokolos]

There is a setting in moonlight I always have on called "never drop frames". Streaming is perferct for me that way and annoying microstuttering (which I see on ALVR sometimes) vanishes. Can you maybe investigate on what it does and maybe replicate the functionality and add the option on the server? It mentions it adds latency but I never had any problem on 2D streams. Don't know how it will affect async timewarp though.

[polygraphene]

I fixed some bug on adaptive IDR insertion on release v2.1.5. Please check it.

I know "never drop frames" settings but I can't understand what the settings does. Does it really have significant effect? https://github.com/moonlight-stream/moonlight-android/commit/5e5df8abc8546d2e4e7dbad5f5869251fa031f83#diff-5747d6ac2ccd1eef5676ae2f5aaf4e67R369

[vrokolos]

I will test new version tonight.

About never drop frames: Whenever I use moonlight from my TV or android phone the motion is not so smooth. There is a stutter that happens all the time. Whenever I use that option everything becomes extremely smooth.

I've seen such stutter on ALVR on fast moving objects. The motion is not that smooth even with 60 fps. I don't know if that option would help though.

https://stuff.mit.edu/afs/sipb/project/android/docs/reference/android/media/MediaCodec.html

It seems to only change the 2nd parameter Boolean on ReleaseOutputBuffer function from true to false.

render | If a valid surface was specified when configuring the codec, passing true renders this output buffer to the surface.

[vrokolos]

https://github.com/polygraphene/ALVRClient/blob/4c1b686cdb3a7f31a1545b7746bf828e340e2715/app/src/main/java/com/polygraphene/alvr/VrThread.java#L409

https://github.com/polygraphene/ALVRClient/blob/6ca923919fcff491e48cf7c6dc74127b9ac59226/app/src/main/java/com/polygraphene/alvr/DecoderThread.java#L305

https://github.com/polygraphene/ALVRClient/blob/6ca923919fcff491e48cf7c6dc74127b9ac59226/app/src/main/java/com/polygraphene/alvr/DecoderThread.java#L279

You're using it with both true / false. I don't understand though :) I've never seen such stutter while moving head. Only on objects

[polygraphene]

Thank you!

I found the best matching document. https://developer.android.com/reference/android/media/MediaCodec#releaseOutputBuffer(int,%20long)

If you render your buffer on a SurfaceView, you can use the timestamp to render the buffer at a specific time ... if the timestamp is not "reasonably close" to the current system time, the Surface will ignore the timestamp, and display the buffer at the earliest feasible time. In this mode it will not drop frames. for best performance and quality, call this method when you are about two VSYNCs' time before the desired render time. For 60Hz displays, this is about 33 msec.

I got the meaning of "never drop packet". This options force the rendering frame timing into VSync. And this will reduce duplicate frame on a single VSync or frame skip by the jitter of frame arrival time.

Moonlight passes "0" which is not "reasonably close" to the current system time. And which enable this feature on SurfaceView.

ALVR client does not use SurfaceView, so we should implement this feature by ourselves.

BTW, "0" does not match boolean on Java, so this calls other overloaded methods:-)

[vrokolos]

BTW, "0" does not match boolean on Java, so this calls other overloaded methods:-)

WTF!!

[vrokolos]

I fixed some bug on adaptive IDR insertion on release v2.1.5. Please check it.

I tested this version and it works far better than anything else I tried. Also tried using 200mbit/s on skyrim and h264 becomes wonderful on 200mbit/s. Consider raising the max bitrate!

https://github.com/polygraphene/ALVR/pull/80

[polygraphene]

Great! I have never heard that high bitrate... I will test it.

[vrokolos]

I've got an ac router and the benchmarks I do on the wifi speed shows that it is around 280mbits/sec

[polygraphene]

That was too heavy for my 11n router. But I will merge your PR. Thank you!

[dusan-t]

@vrokolos

Was there a difference only in image quality or was the micro-stuttering also gone and how much of that difference do you think is due to the bug fix in adaptive IDR in version 2.1.5 and how much due to increase in speed ? Have you also tried with 100 Mbit/s, is it any better than before the fix ? Thanks.

Btw, if you would like to do a better comparison between SteamVR in Moonlight and SteamVR in ALVR, you can do it with Riftcat/Vridge, it supports both Oculus Go and Moonlight. There is also Moonlight for Go, but it doesn't do SteamVR, only 2D and 3D SBS but if nothing else it will allow you to view the Moonlight stream from inside of the Go instead of just looking at the image on a regular phone or tablet screen.

[vrokolos]

@dusan-t

The microstuttering is there and I'm pretty sure it's cause of the vsync thing mentioned above. No fps change or network change can fix that.

The new version made the quality a bit better on the same bandwidth and it fixed a lot of the packet loss handling.

Getting from 100mbit to 200mbit made the quality of dark moments a lot better. Artifacts almost vanished.

During bright moments everything works great even on 20mbit. But on skyrim and during the night I could see artifacts when on 100mbit. On 200mbit I think it's quite better.

You mentioned moonlight SBS. I can't find it for my Gear VR and looking for it. Please tell me how to do moonlight SBS

[dusan-t]

Hm, I was under the impression that you had an Oculus Go and it looks like this doesn't work on GearVR - developer wrote they heard from a few people that it didn't work for them on GearVr but you may try it just in case. As for StreamTheater which was an excellent solution back then, it seems it's now dead as well.

Moonlight for Go: https://www.reddit.com/r/OculusGo/comments/8mc64q/moonlight_for_go_stream_your_pc_gamesdesktop_to_a/

And the Git page of the project: https://github.com/penkamaster/OculusSdk-moonlight

[dusan-t]

I'm sure you're aware of this, but just in case, you can always use regular Moonlight on your Samsung phone, but it won't be relaying on Gear's sensors for tracking and for SBS you could use Tridef or 3D Vision to make regular games 3D SBS. Oh yeah, if you can make your phone Daydream compatible, there is also Moonlight for Daydream from the same author: https://www.reddit.com/r/daydream/comments/877mcp/moonlight_for_daydream_stream_your_pc/ I'm mentioning this because Daydream's tracking is supposed to be pretty good, maybe not exactly as Gears but close.

[dusan-t]

Btw, I'm using regular Moonlight on my Android phone in combination with Tridef and SteamVR (through one other solution, not Riftcat) and I don't have the 'never drop frames' box turned on but I'm not experiencing micro stuttering, at least not while everything is working fine. Occasionally though, I do notice stutter but I think that's due to heavy scenes that my PC can't handle correctly or/and due to tracking setup (phone + opentrack)

[vrokolos]

@dusan-t Then I might be wrong. Maybe forcing vsync on from nvidia control panel affects this? I shoukd try that some time. Anyway on my android everything is smoother with that setting on and I dont experience any latency . I'm talking about moonlight 2d.

[dusan-t]

I'm definitely going to try 'never drop frames' option, although I do have adaptive vsync turned ON in Nvidia panel. I will play with the settings some more as maybe I wasn't paying enough attention to stuttering and got used to inferior experience, or maybe those occasional bigger stutters will decrease.

[vrokolos]

If you've got a router that can do more than 200mbit/s do it. It's worth it. Dark areas become so much clearer!!

Also raising my buffer to 1.5mb I don't get any packet loss any more. This is awesome. Now I can't wait for reduced microstutter.

[vrokolos]

Changing my router wifi preamble mode from long to short and changing the wifi channel made things even better on packet loss. Now it seems the only broken image/packet loss is related to CPU issues on the server since framerate also drops when it happens. Maybe a key frame is dropped when the CPU is overloaded?

Tried experimenting with forcing VSYNC on nvidia control panel but the really annoying jitter/stuttering is still there. I think it can only be fixed by @polygraphene on the client doing that vsync thing above.

Also tested ARCore again and I'm pretty sure I won't use it again. It didn't work that well. There is too much lag / drifting. It's not ALVR's fault and it's not worth it for me to mess around with it more . I love playing seated with a gamepad.

[vrokolos]

Found this thread talking about the same microstuttering on moonlight:

https://github.com/moonlight-stream/moonlight-android/issues/349

They talk about "never drop frames" and a hacked 59fps mode to reduce jitter. Very interesting.

Gonna test 59fps streaming tonight!

[vrokolos]

Changing to 59 fps didn't do a thing. Probably didn't change it everywhere. Changed it only on server side.

[vrokolos]

With v2.2.2 there is a massive difference in broken images. Everything works perfectly now. There are literally no broken images even with some percent packet loss. I'd consider the packet loss issue fixed!

[polygraphene]

Thank you for trying! Still has some bugs, but I'm glad it works.