Possibility of running TurboVNC over UDT/QUIC

[nEghbal]

Hello,

Is it possible to replace TCP with UDT or QUIC to connect the VNC client to the server?

Thanks.

[dcommander]

Unfortunately not. RFB doesn't have any inherent error correcting, error handling, or message ordering, so it depends on the underlying transport protocol to do that. Thus, it would never work with UDP. Is there a specific reason why you need UDP?

[nEghbal]

Does RFB require ordering within each update?

[dcommander]

I’ll have to think about that one in depth. The framebuffer updates themselves must arrive in order, but possibly the order of rectangles within a given framebuffer update can be changed. You still haven’t told me what your ultimate goal is with this. The more I know about what you’re trying to accomplish, the better my ideas will be. It may be that there is a solution to the problem that doesn’t involve UDP.

[nEghbal]

I wanted to study the impact of releasing the total ordering of rectangles within each update on the performance of VNC. To do that, I need to switch to an underlying protocol which allows this like UDP-based Data Transfer (UDT) or QUIC.

[nEghbal]

I also had another question. In the main page you said "TurboVNC, in combination with VirtualGL, provides a complete solution for remotely displaying 3D applications with interactive performance." but in the introduction part of the TurboVNC website you said "Double buffering on the client side to reduce tearing artifacts in 3D and video applications".

Q: with double buffering how TurboVNC can provide a complete solution for remotely displaying 3D applications with interactive performance?

Thanks.

[dcommander]

I wanted to study the impact of releasing the total ordering of rectangles within each update on the performance of VNC. To do that, I need to switch to an underlying protocol which allows this like UDP-based Data Transfer (UDT) or QUIC.

I don't see how that would affect performance unless you could send and decode the rectangles in parallel somehow, using multiple data streams. I do eventually intend to implement multithreaded decoding, as TigerVNC does, but multithreaded decoding with a single data stream is a mixed bag in terms of performance. In TigerVNC, it slows down the single-threaded case, which is why I have been hesitant to adopt it. I want to figure out how to implement it in a way that minimally does no harm.

When you start talking about changing the underlying protocol and using multiple data streams and such, you are no longer talking about VNC, because none of those changes would be in any way compatible with other VNC solutions. You would essentially have to create a whole new protocol. TurboVNC's performance on modern hardware is mostly limited by compression and decompression on gigabit networks and by network bandwidth on 10 Mbit and smaller networks, so I remain unconvinced that UDP could improve upon that unless you were running over a really unreliable network and you could somehow implement error correction at the RFB protocol level. Again, however, that would no longer be the RFB protocol. It would be an incompatible derivative of the RFB protocol. From my point of view, implementing new compression methods is much lower-hanging fruit in terms of improving performance over low-bandwidth networks, and implementing multi-threaded decompression is much lower-hanging fruit in terms of improving performance over high-bandwidth networks.

I also had another question. In the main page you said "TurboVNC, in combination with VirtualGL, provides a complete solution for remotely displaying 3D applications with interactive performance." but in the introduction part of the TurboVNC website you said "Double buffering on the client side to reduce tearing artifacts in 3D and video applications".

Q: with double buffering how TurboVNC can provide a complete solution for remotely displaying 3D applications with interactive performance?

VirtualGL adds GPU acceleration for OpenGL applications. Without VirtualGL, TurboVNC uses software OpenGL rendering, which is quite sluggish (particularly with large 3D geometries) and can create CPU competition with the TurboVNC image compressor, thus reducing the frame rate to below "interactive" levels.

Double buffering in TurboVNC is unrelated to VirtualGL. What it means is that TurboVNC waits until the entire framebuffer update has been decoded before displaying it. TurboVNC was, in fact, the first VNC solution to do that, which is why I saw fit to advertise it, but I think all of them do that these days. The very first implementations of TurboVNC released in 2004, which were essentially just naive ports of TightVNC 1.3 with a SIMD-accelerated JPEG codec, lacked double buffering, so there was visible flicker when VirtualGL or video players drew frames in rapid succession.

[nEghbal]

Thanks for your detailed response.

I am a PhD student and working on ideas to improve the performance of latency-sensitive workloads over wide-area networks which have high RTT. We recently published a paper in ICCS 22 about reducing the inter-arrival time of messages using a synthetic workload. Now, I am looking at VNC as a candidate real workload. Using UDT (not UDP) or QUIC as the underlying protocol, I do not need to worry about packet loss because they are reliable. But I can release the total ordering within each update to reduce the lag/blocking when a loss happens specially in high RTT networks in which waiting for a retransmission takes a noticeable time. We have the consistency fence idea to specify the order of updates and we showed using Forward Error Correction (FEC) can help further finishing the whole update and starting the next sooner.

I think using consistency fences and FEC would not be "incompatible derivative of the RFB protocol" but it is the way of running it over something other than TCP. TurboVNC and TigerVNC had previously modified the RFB protocol by sending more frequent updates than the original RFB which was demand driven.

One point about double buffering: running over TCP, when there is a packet loss in any point of the update, the rest will be blocked, let's say half of the screen on average. But running over UDT/QUIC, when we can deliver all the rectangles within each update while waiting for the retransmissions or FEC recovery, there would be only a few number of lost rectangles which won't be showing and may not bother the user.

[dcommander]

That all sounds very interesting. I confess that I don't fully understand the ramifications from the point of view of the VNC server/viewer, however. Would this replace the existing flow control implementation (which we borrowed from TigerVNC)?

[dcommander]

Also, could you take advantage of the existing RFB fence extension, which allows arbitrary data to be sent in the fence payload?

[nEghbal]

That all sounds very interesting. I confess that I don't fully understand the ramifications from the point of view of the VNC server/viewer, however. Would this replace the existing flow control implementation (which we borrowed from TigerVNC)?

No, I think to see the best performance we need to send more frequent updates using a flow/congestion control algorithm.

[nEghbal]

Also, could you take advantage of the existing RFB fence extension, which allows arbitrary data to be sent in the fence payload?

I am not sure. I need to look into the code. Do you know if anyone would like to collaborate on these ideas? as I know nothing about the code.

[dcommander]

I can point you in the right direction regarding the TurboVNC code, but I unfortunately don't have time for a full collaboration, and I'm the only person who has any deep expertise with the code.

[nEghbal]

I can point you in the right direction regarding the code, but I unfortunately don't have time for a full collaboration.

That can help a lot. Thanks. I will post my questions then :)

[nEghbal]

Do you know about any more recent evaluations than this study?

[dcommander]

No, this is a single-developer project, so any more recent evaluations would have been done by me. Anyhow, that study was intended to tweak the TurboVNC encoding methods to balance the goals of reducing network and CPU usage. It was conducted using an encoding simulator, so it doesn't take into account high-latency or unreliable networks. The assumption at the time was that, eventually, TurboVNC would evolve to the point at which network latency didn't limit the data throughput. That assumption ultimately became valid in TurboVNC 1.2 four years later, with the introduction of the RFB flow control extensions.