Consider encodings using zlib (zlib, tight, ZRLE)

[kanaka]

Standard tight encoding and ZRLE will require zlib decompression in Javascript. This will probably be pretty slow (based on some initial testing), but having them as (non-default) options might be worthwhile in some cases.

[kanaka]

For general zlib decompression, tobeytailor's experiment in Gordon may be an interesting approach: https://github.com/tobeytailor/gordon/blob/5ebbd7d22076504c4c8e1ccaa6b9849163daf739/src/inflate.js

[kanaka]

Other links that will likely be useful:

• http://en.wikipedia.org/wiki/Portable_Network_Graphics
• http://www.w3.org/TR/PNG/
• http://www.ietf.org/rfc/rfc1950.txt - ZLIB
• http://www.ietf.org/rfc/rfc1951.txt - deflate
• http://lists.w3.org/Archives/Public/public-html-diffs/2009Jun/0082.html - lossy image data

[kanaka]

There may also soon be a LZMA2 encoding soon: https://forum.ultravnc.net/viewtopic.php?f=52&t=26926

[mightypenguin]

I may be completely off here but how hard would it be to have the Websockit proxy transcode to tightPNG from one of these? I assume doing it in Python wouldn't be a huge problem, might add more lag I suppose.

[kanaka]

Websockify has no knowledge of the RFB/VNC protocol. It is purely a bridge/proxy between WebSockets and raw TCP sockets.

You could certainly make a proxy/bridge that understands the VNC protocol and do encoding translations (in which case you could pretty easily translate zlib and regular tight to tightPNG). In addition the the latency, the other issue is that the RFB protocol is not distinctly framed so the proxy/bridge would have to fully understand all the different msg types (even ones it was going to pass straight through) because it would need to know how long those messages were so that it can find the next message in the stream.

Again, it's certainly possible, but it's a fairly non-trivial project and would be a separate program from websockify.

[kanaka]

Well, considering that recent Firefox builds can do full-speed H264 decoding, I suspect that zlib/tight encodings should soon be no problem for noVNC:

• http://www.nsaneforums.com/topic/105614-mozilla-dev-releases-javascript-based-h264-decoder/
• http://shaver.off.net/diary/2011/10/28/approaches-to-performance/

@mightypenguin, any progress on implementing any of the zlib based encodings?

[mightypenguin]

No real progress. Just cleaned up view_only so hopefully that's acceptable now. Will probably end up working zlib stuff at work but lots of other fires to put out so no timeframe.

[mightypenguin]

Worked on this zlib thing and stuck atm. Been trying to use dankogai (https://github.com/dankogai/js-deflate) with help from http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding

So for every call to encHandlers.ZLIB (my creation) and then data I get inside that from websock.rQshiftBytes to give me 4 bytes length followed by that length bytes zlib stream which I pass directly to the inflater.

The initial block of around 9-10kb seems to decode properly, and displays as 32 black lines at the top of the screen. But the next 1 or 2 blocks later is 32/64bytes which inflates to around 615ish bytes which makes no sense as a raw type draw. So it's something extra. I tried just ignoring that and grabbing the next block, but obviously bad things happen.

I can see that the server keeps sending more data cause the buffer get's bigger.

Fedora's tigervnc supports straight zlib protocol so I've been using that for debugging. So I've been slowly perusing that... http://tigervnc.svn.sourceforge.net/viewvc/tigervnc/branches/1_1/common/rfb/

I've asked for help on their mailinglist so we'll see what happens.

32/64 block always starts with {4 0 0 32 0 0 0 6 0 0 }

[kanaka]

@mightypenguin, looks like you are making good progress. If you can post your current code to your noVNC fork in a branch I could take a look.

BTW, the license for dankogai/js-deflate doesn't permit me to include it in noVNC. If you follow the history back to where the code came from, it is here: http://www.onicos.com/staff/iz/amuse/javascript/expert/. The other Javascript utilities have a license that is compatible with noVNC. Unfortuantely, deflate.js and inflate.js are simple copyright to Masanao Izumo with no license (and no explicit license means all rights are reserved to him).

There are probably other JS inflate implementations out there. Or you can try and get those files licensed in a way that noVNC could use. noVNC is LGPL-3, so it can use incorporate code using most other more permissive open source licenses (apart from GPL), such as BSD, MIT, Apache, Mozilla and other LGPL-3. However, even if Masanao Izumo gave his permission, those files are hosted on a commercial site and I would question whether he in fact owns the copyright directly, or whether it was a work for hire and therefore not actually his copyright. So my recommendation would be to find an alternate inflate implementation in JS and use that.

[mightypenguin]

Well that's too bad. His code actually handles zlib in stream/dynamic mode, a couple other ones I've found only handle gzip files. Could try to use a translator from the zlib source code but that will be pretty opaque to debug/optimize.

[kanaka]

Yes, his is the easiest to find and a lot of other implementations are really just re-packaging of the same code.

A couple of other projects I've seen that have independent implementations are:

• https://github.com/tobeytailor/gordon/blob/master/src/stream.js
• http://jsxgraph.uni-bayreuth.de/wp/2009/09/29/jsxcompressor-zlib-compressed-javascript-code/

I haven't looked at them in detail to see how difficult the plain inflate/deflate code would be to extract and use.

[mightypenguin]

jsxgraph appears to not support streaming. gordon's code is giving me some trouble.

Can you explain or link a tutorial on how to use code that's inside gordon's "(function() { ...." Still catching up on some of this oop stuff and apparently I'm not using the right search terms. It would make sense to me if it was "namespace = function() { ..." but how are you supposed to reference code outside that function if it doesn't have a name?

[kanaka]

The (function() {})() syntax is Javascript's only good way of doing lexical scoping. If you see it without it being assigned to anything then this means that it is being used for side effects (e.g. to set global variables). In this case it is defining the object Gordon.Stream.

You can see Stream being used in https://github.com/tobeytailor/gordon/blob/master/src/parser.js in particular search for "s.decompress()".

[kanaka]

Just saw zip.js which contains inflate and deflate implementations. Might work for noVNC: https://github.com/gildas-lormeau/zip.js/blob/master/WebContent/inflate.js

[mightypenguin]

That does look really promising!

Right now going to 8bit color has solved most of my bandwidth problems, but I might still play with this at some point if I want to go back to truecolor.

[quicklyfrozen]

I found a zlib implementation at https://github.com/operasoftware/jsunzip and used it to implement Tight support (truecolor only) in my fork: https://github.com/quicklyfrozen/noVNC

[kanaka]

@quicklyfrozen, excellent! I've tested it and it is working for me and is surprisingly fast.

One thing I notice right away is that the new encoding is messing with the timing/performance tracking mechanism in noVNC. At least with tightvnc, the rectangle count in the FBU frame is very high with tight (probably because the server doesn't know beforehand how many rects it's going to send). Not a blocking issue of course, but it makes quantitative performance comparisons difficult.

Hopefully I'll get a chance to look deeper this weekend.

Also, I notice in your repo that you have socket.io changes as well. Have you gotten that working? And if so, have you seen much latency difference when using a non-WebSocket transport?

[quicklyfrozen]

@kanaka: If you turn off last_rect encoding (by commenting out line 66) I expect the rectangle counts will be more sane.

Yes, I have got the socket.io transport working. It works fine if it uses the Flash emulation :-). I'll have to do more testing to see how the latency is when it falls back to long polling. I also need to extract the Node.js "proxy" code so I can publish it (it's quite simple, but it's mixed in with some other code).

BTW, thanks for this great project.

[kanaka]

@quicklyfrozen, I merged and pushed out your tight encoding changes. I didn't merge the changes related to socket.io. However, I'm still definitely interested in that once you are further along and have published the server side code too.

I decided to make the last rect encoding handler more like the rest instead of being a special conditional in the main framebufferupdate handler. I also changed the way that it determines if the first full frame has been rendered by counting pixels (in addition to rects being 0).

Good work on the tight encoding and on finding the excellent and performant jsunzip library. I did a little code formatting cleanup, but otherwise it was good clean code. And I'm pleasantly surprised at how well it performs (at least on recent Chrome and Firefox, I haven't done testing of it on older browsers yet).

In the future I'll probably merge the tight and tightPNG handlers since there is a lot of code overlap (tightPNG is a modification of tight anyways).

[kanaka]

I consider this issue complete. Now that we have a good inflate/zlib library, the zlib and zrle encodings should be pretty easy. I will accept implementations of those if somebody wants to do them, but they are pretty low on my priority list though so I'm closing this bug.

[leslie-qiwa]

@kanaka did you merge tight_png part? @quicklyfrozen seems like rfb.js has changed a lot. so would you mind to update your tight_png implementation with upstream?

Background, VMware ESXi VNC uses tight_png encoding. It can decode if forcing to use old tight, it can work, but have graphic glitches sometimes. So I'm thinking it would be great if someone can add the support.

[CendioOssman]

TightPNG has been in noVNC since c0c20581f5598a7e0b864d20591e7f65f17c3445 and the protocol has not changed since then AFAIK. There might be bugs though since very few servers use that encoding.

[kanaka]

@leslie-qiwa Can you point to any documentation showing that VMware ESXi supports tightPNG? I'm not able to find anything with some quick googling but I would be interested to see some evidence if you have it. I developed tightPNG (along with Corentin Chary) and last I knew it was only support by QEMU and libvncserver.

[leslie-qiwa]

No document shows that. I found it because noVNC reports error "unknown encoding -260".

I use https://github.com/amitbet/vncproxy to dump all messages. It shows ESXi returns both Tight and TightPNG.

Please refer this issue https://github.com/novnc/noVNC/issues/1023

[leslie-qiwa]

https://github.com/novnc/noVNC/pull/1026 is created. It works in our ESX environment.

[CendioOssman]

Ah, I see the problem. I've opened #1027 so we can handle this discussion better.