Latency?

[lattice0]

Hi, I'd like to know if this project is enough for local graphical development over wifi 6, to another computer of mine running Android Studio with Android Emulator. I'm searching for something with less latency than VNC, since it's all local. Do you have like a video of such type of local usage so I can see if it's usable?

[serpilliere]

Hi @lattice0 Here is a video recorded at 60 fps from a fullhd screen. The servers encodes here on intel default gpu using QSV The client decodes as well on intel using qsv.

https://github.com/cea-sec/sanzu/assets/9433661/e34348d6-5da4-4f77-bb00-e53a010b8a96

it seems that the video I recorded are not shown here (marked as corrupted ?!) , but you can download it at play it. I recorded it at 60 fps to be as close as the reality

Note that on the current branch of sanzu, the latency, even if acceptable, can be optimized. We are currently working on it with an 'async' version of sanzu (and may be released soon)

[lattice0]

Can't watch the video. Is it local or remote connection? I'd use it mosly locally and over wifi 6 to give less latency (quality/colors are not priority). Context: I'm tired of super hot amd64 laptops with 2h battery life and using a Galaxy Tab S9 Ultra for coding now in vscode.dev, but for Android Development I'd possibly need the Android Studio + Emulator. I understand though it would take time for an Android app to be developed for this project so I can use on the galaxy tab S9 ultra, but I'm just dreaming for now. Maybe I could help in the future, I did lots of Rust on Androd, specially interfacing with ffmpeg and doing yuv -> RGB conversion on GPU using shaders instead of CPU.

What about better performing codecs so we have less latency? Would be nice to have low latency. RustDesk is mostly focused for IT support, I always wanted a open source project that mimics Steam Link, but just for coding, don't need extreme quality, just low latency. The only one I found was this project. I wish sucess.

[serpilliere]

The video is recorded locally (As I said, you should be able to download & play it: the preview on github seems broken) I tried to convert it to gif, but fails at geenrating 60fps gif.

The latency is reduced today if you encode using performant hardware: We have tested it with Nvidia Tesla T4 and Nvidia A16: our observations are those encoders actually encode a frame (including the rgb to yuv using ffmpeg rgb0 input for h264_nvenc) in about 6ms. The decoding part is about 3.3ms for fullhd frame, plus about 1.6ms to turn yuv to rgb.

[lattice0]

Couldn't play it by downloading as well

-------- Mensagem Original -------- Em 4 de out. de 2023 03:17, serpilliere escreveu:

The video is recorded locally (As I said, you should be able to download & play it: the preview on github seems broken) I tried to convert it to gif, but fails at geenrating 60fps gif.

The latency is reduced today if you encode using performant hardware: We have tested it with Nvidia Tesla T4 and Nvidia A16: our observations are those encoders actually encode a frame (including the rgb to yuv using ffmpeg rgb0 input for h264_nvenc) in about 6ms. The decoding part is about 3.3ms for fullhd frame, plus about 1.6ms to turn yuv to rgb.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[serpilliere]

Ok! Here is the video (it was encoded ing yuv444 but browsers only support yuv420, sorry for that)

https://github.com/cea-sec/sanzu/assets/9433661/7e7c0c20-870e-40d0-9f23-7d535da5038c

On a local run of Xeyes, the cursor is on the middle of the pupil of the eye. So here, we can observe latency when the pupil lags behind the cursor. Again, that's using intel integrated gpu for encoding.

[lattice0]

That's fully usable for development 👀 I'll keep an eye on this project. Unfortunately my use case would be with an Android client

[serpilliere]

The async version we are working on is based on SDL2 for now: If I correctly read SDL2 features, this should allow to port it to Android more easily. In this version, we have successfully tested yuv2rgb using graphics shaders and opengl rendering. The result is interesting as most computation are now offloaded to the GPU, which handles this without problems.

[lattice0]

So, it's possible to open individual windows? This would be very cool on Android, I could have Android Studio as a window

[serpilliere]

As explained in the readme, Sanzu supports seamless mode (see main page) I don't know if it will work on Android.