vr headset support

[comsyspro]

Is your feature request related to a problem? Please describe. Would it be possible to use moonlight to stream VR?

Describe the solution you'd like On the remote GPU system you can start a VR app which produces a side-by-side video stream in fullscreen. This video stream should be transfered to the moonlight client. The side-by-side video should then be splitted to a connected VR headset on the client side.

Any ideas how to do that and would it be possible with moonlight? How to split the videostream for the left and right eye in the VR headset?

Is it possible that you can recieve the videostream and directly transfer it to the headset, so that it is not shown on the client monitor?

Would this be a starting point?

import cv2
import openvr

def initialize_vr():
    openvr.init(openvr.VRApplication_Scene)
    vr_system = openvr.VRSystem()
    return vr_system

def shutdown_vr():
    openvr.shutdown()

def main():
    # RTSP Stream URL (is it possible to grab a sunshine stream like that?)
    rtsp_url = "rtsp://username:password@ip_address:port/path"

    # Connect to the RTSP stream
    cap = cv2.VideoCapture(rtsp_url)
    if not cap.isOpened():
        print("Error opening RTSP stream")
        return

    vr_system = initialize_vr()

    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            print("Error reading frame")
            break

        # Display the frame (optional for checking)
        cv2.imshow('RTSP Stream', frame)

        # Send the frame to the VR headset
        # You need to insert your VR rendering logic here, such as rendering the frame as a texture
        # This is a placeholder, as the actual implementation depends on the VR headset and VR library

        # Wait for the 'q' key to be pressed to exit
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    # Release the capture objects and close the windows
    cap.release()
    cv2.destroyAllWindows()
    shutdown_vr()

if __name__ == "__main__":
    main()

[thesleepiestdev]

i believe Virtual Desktop Classic may be able to help you with this https://store.steampowered.com/app/382110/Virtual_Desktop_Classic/

enter fullscreen on moonlight, and this will let you bring your feed into VR and correctly map the halves of the screen to each eye. i'm not sure if it's perfect though

[comsyspro]

I looked into it, but this is not the solution. I want somehow forward the videostream from sunshine from the remote to the local computer. For that there should be a function in moonlight which can recieve this videostream (stereo side by side) and forward it to the VR headset. The transfering of sensor data is also another topic which would have to be solved. So it's not an easy task and it would be a good feature for VR gaming.