Instruction to test

[muratmaga]

I am testing this repository for remote desktop with GPU, using this g3.medium from https://docs.jetstream-cloud.org/general/vmsizes/#jetstream2-gpu.

This instance has Nvidia driver 525.60.13. As per support matrix, this gpu (A100) doesnt support NVENC, so I replaced WEBRTC_ENCODER with vp9enc. Here is the full docker command:

docker run --gpus 1 -it --tmpfs /dev/shm:rw -e TZ=UTC -e SIZEW=1920 -e SIZEH=1080 -e REFRESH=60 -e DPI=96 -e CDEPTH=24 -e PASSWD=mypasswd -e WEBRTC_ENCODER=vp9enc -e BASIC_AUTH_PASSWORD=mypasswd -p 8080:8080 ghcr.io/selkies-project/nvidia-egl-desktop:latest

here is the output of the this:

2023-01-19 19:32:05,876 INFO Set uid to user 1000 succeeded
2023-01-19 19:32:05,878 INFO supervisord started with pid 1
2023-01-19 19:32:06,882 INFO spawned: 'entrypoint' with pid 7
2023-01-19 19:32:06,885 INFO spawned: 'pulseaudio' with pid 8
2023-01-19 19:32:06,888 INFO spawned: 'selkies-gstreamer' with pid 9
2023-01-19 19:32:06,898 INFO reaped unknown pid 15 (exit status 1)
2023-01-19 19:32:06,980 INFO reaped unknown pid 21 (exit status 1)
2023-01-19 19:32:06,980 INFO reaped unknown pid 24 (exit status 1)
2023-01-19 19:32:06,980 INFO reaped unknown pid 27 (exit status 1)
2023-01-19 19:32:06,981 INFO reaped unknown pid 31 (exit status 1)
2023-01-19 19:32:06,981 INFO reaped unknown pid 36 (exit status 1)
2023-01-19 19:32:06,981 INFO reaped unknown pid 39 (exit status 1)
2023-01-19 19:32:06,981 INFO reaped unknown pid 43 (exit status 1)
2023-01-19 19:32:06,988 INFO reaped unknown pid 46 (exit status 1)
2023-01-19 19:32:07,900 INFO success: entrypoint entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-01-19 19:32:07,900 INFO success: pulseaudio entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-01-19 19:32:07,901 INFO success: selkies-gstreamer entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-01-19 19:32:07,931 INFO reaped unknown pid 65 (exit status 1)
2023-01-19 19:32:07,931 INFO reaped unknown pid 62 (exit status 1)
2023-01-19 19:32:07,931 INFO reaped unknown pid 69 (exit status 1)
2023-01-19 19:32:10,103 INFO reaped unknown pid 144 (exit status 0)
2023-01-19 19:32:10,239 INFO reaped unknown pid 151 (exit status 0)
2023-01-19 19:32:10,752 INFO reaped unknown pid 165 (exit status 0)
2023-01-19 19:32:10,962 INFO reaped unknown pid 254 (exit status 0)
2023-01-19 19:32:12,550 INFO reaped unknown pid 333 (exit status 0)

When I login with the provided user name and password, things seem to stuck at the "Waiting for video stream" step. I am not getting any desktop, even if I wait for 10 minutes. Am I missing some steps from instructions?

And these are the status logs from the client, if they are of any help:

[11:33:17] [webrtc] [ERROR] attempt to send data channel message before channel was open.

[11:33:19] [signalling] Connecting to server.

[11:33:21] [signalling] Registering with server, peer ID: 1

[11:33:23] [signalling] Registered with server.

[11:33:23] [signalling] Waiting for video stream.

[11:33:25] [webrtc] [ERROR] attempt to send data channel message before channel was open.

[11:33:25] [webrtc] Received incoming video stream from peer

[11:33:25] [webrtc] Received incoming audio stream from peer

[11:33:25] [webrtc] Completed ICE candidates from peer connection

[11:33:43] [webrtc] [ERROR] attempt to send data channel message before channel was open.

[11:33:46] [webrtc] [ERROR] attempt to send data channel message before channel was open.

[11:33:46] [webrtc] [ERROR] attempt to send data channel message before channel was open.

[11:33:48] [webrtc] [ERROR] attempt to send data channel message before channel was open.

[muratmaga]

it worked with ENABLE_NOVNC.

[ehfd]

Academia! Hello. If a TURN Server is set up, it can work on Jetstream as well. It can also work if you expose enough UDP ports on OpenStack.

[ehfd]

In the future, our goal is to use the QUIC capacity of recent web browsers to remove that constraint.

[muratmaga]

Thank you. Do you have any performance comparisons for gstreamer vs novnc? Our intended use is realtime 3D visualization and segmentation of large 3D scan dataset on Jetstream.

[ehfd]

Thank you. Do you have any performance comparisons for gstreamer vs novnc? Our intended use is realtime 3D visualization and segmentation of large 3D scan dataset on Jetstream.

GStreamer: Stable 1080p 60 fps or higher, can use NVIDIA GPUs noVNC: You are very lucky if you can reach 15-20 fps...

[muratmaga]

Would you be able to advise us on how to set GStreamer up correctly? A100 GPUs really powerful, but would the lack of nvENC is a concern?

[ehfd]

Would you be able to advise us on how to set GStreamer up correctly? A100 GPUs really powerful, but would the lack of nvENC is a concern?

Argh... A100. This is a GPU that I specifically do not advise for graphical workloads. A40/A10 (of which we mainly use for the National Research Platform NSF project) are fine. But there is x264enc or vp8enc and a good CPU will cover for that NVENC deficit.

[ehfd]

I will ask the GStreamer team if the new https://gstreamer.freedesktop.org/documentation/nvcodec/nvcudah264enc.html interface works on A100.

[ehfd]

But even so, this plugin will be available on GStreamer 1.22, not the current 1.20.

[ehfd]

References: https://ucsd-prp.gitlab.io/userdocs/running/gui-desktop/ I work with the DeFanti group of the UCSD Qualcomm Institute and the National Research Platform. Jetstream is definitely something we can try our best to help on.

[muratmaga]

Thank you. Do you have any performance comparisons for gstreamer vs novnc? Our intended use is realtime 3D visualization and segmentation of large 3D scan dataset on Jetstream.

GStreamer: Stable 1080p 60 fps or higher, can use NVIDIA GPUs noVNC: You are very lucky if you can reach 15-20 fps...

What about the bandwidth usage? FPS is not really all that critical (as long as we get 8-10fps, that's totally usable for our purposes). However, mouse interactions and other lag was one of the issues with VNC based solutions, particularly on slow networks.

[muratmaga]

References: https://ucsd-prp.gitlab.io/userdocs/running/gui-desktop/ I work with the DeFanti group of the UCSD Qualcomm Institute and the National Research Platform. Jetstream is definitely something we can try our best to help on.

What would be the best way to reach out for help? use this issue tracker or is there a contact we can use?

[ehfd]

What about the bandwidth usage? FPS is not really all that critical (as long as we get 8-10fps, that's totally usable for our purposes). However, mouse interactions and other lag was one of the issues with VNC based solutions, particularly on slow networks.

Bandwidth of Selkies WebRTC is configurable in the web interface. For 1080p, 3-4mbps should be fine on 30 fps and 8mbps-12mbps on 60 fps is optimal. I could "use" it on a cellular connection capped to 1mbps with 500kbps and 15 fps, but that is too extreme. Mouse interaction actually gets better as fps improves.

Instead of using vncserver -> noVNC or libvirt/OpenStack -> noVNC, try using Xvfb/Xorg -> x11vnc -> noVNC. I have had not much issues on that and it's pretty smooth. My container provides an example of that with NOVNC_ENABLE=true.

What would be the best way to reach out for help? use this issue tracker or is there a contact we can use?

Our engineers at UCSD operate a Matrix instance. Tag @ehfd in https://matrix-to.nrp-nautilus.io/#/#ue4research:matrix.nrp-nautilus.io or https://matrix.to/#/#ue4research:matrix.nrp-nautilus.io according to the below instructions.

https://ucsd-prp.gitlab.io/userdocs/start/contact/