Add TightPNG encoding

[sidorares]

http://wiki.qemu.org/VNC_Tight_PNG

[vtolstov]

i think this is more proper place. Now i need only working compact length calculation for tight png working. I found simple code from wireshark dissector for tight encoding http://code.metager.de/source/xref/wireshark/epan/dissectors/packet-vnc.c#3031

i'm already check my own code and it looks the same. So i'm stuck now. sorry for reindent issues =( . i'm cleanup pr when it will be ready https://github.com/sidorares/node-rfb2/compare/master...vtolstov:master?expand=1

[vtolstov]

As i debug, png data in rectangle in my case started after 0,0,0,0,0,0,19,0,231,0,13,255,255,254,252,160,224,2

first rect has rect.x 0 rect.y 0 rect.width 1024 rect.height 19

[sidorares]

I'll be able to help early tomorrow ( +9 hours in my time)

[vtolstov]

No problem, now i have free time and may be found some own mistakes

[vtolstov]

@sidorares so i think that we have error in decoding stream. In case of raw encoding, do you have proper image received from frame buffer update message?

[sidorares]

what I see is that first byte is 0 and reading stops because it's not tight.png

        var tight = rfb.subEncodings.tight;
        console.log('TIGHT: byte 0', ctl[0]);
        ctl = ctl[0] >> 4;
        if (ctl & tight.png) {
            console.log('Reading tight png...');
            cli.readTightPng(rect, cb);
        }

[vtolstov]

Does it possible this issue with endianes? or all rectangles and encoding always send via little endian?

[sidorares]

no, I think the issue is that we can't just add png part only. We need to decode BasicCompression bytes ( or at leas skip them ) - see BasicCompression part here - https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#id78

[vtolstov]

But basic compression part goes after control part...

[vtolstov]

And we have control = 0...

[sidorares]

When I connect first update seems to be in tight / BasicCompression - same color rectangle, I think control byte is followed by single TPIXEL value ( I guess it 3 bytes )

The Tight encoding makes use of a new type TPIXEL (Tight pixel). This is the same as a PIXEL for the agreed pixel format, except where true-colour-flag is non-zero, bits-per-pixel is 32, depth is 24 and all of the bits making up the red, green and blue intensities are exactly 8 bits wide. In this case a TPIXEL is only 3 bytes long, where the first byte is the red component, the second byte is the green component, and the third byte is the blue component of the pixel color value.

[sidorares]

But basic compression part goes after control part...

isn't control = 0 indication of basic compression?

[vtolstov]

@sidorares yes, but why we have tpixel?

[sidorares]

as far as I understand, basic compression + fill = one single pixel follows, same color for whole rectangle

[vtolstov]

so i need to add to tight some compression types like fill, jpeg, basic compression types and recheck control for it ?

[vtolstov]

like

exports.tightCompressions = {
    fill: 8,
    jpeg: 9,
    png: 10,
};

[sidorares]

yes:

If bit 6 of the compression-control byte is set to 0 (no filter-id byte), then the CopyFilter is used.

CopyFilter When the CopyFilter is active, raw pixel values in TPIXEL format will be compressed. See below for details on the compression.

...

After the pixel data has been filtered with one of the above three filters, it is compressed using the zlib library. But if the data size after applying the filter but before the compression is less then 12, then the data is sent as is, uncompressed. Four separate zlib streams (0..3) can be used and the decoder should read the actual stream id from the compression-control byte (see [NOTE1]).

[sidorares]

plus basic

exports.tightCompressions = {
    basic: 0
    fill: 8,
    jpeg: 9,
    png: 10,
};

and basic is CopyFilter, PaletteFilter, GradientFilter

[vtolstov]

@sidorares how we can represent tpixel inside rfb2 package ? create new type ?

[sidorares]

I'd rather just store as node Buffer

[vtolstov]

RfbClient.prototype.readTight = function(rect, cb)
{
    var stream = this.pack_stream;
    var cli = this;

    stream.unpack('C', function(ctl) {
        var comp = rfb.tightCompressions;

        cmp = ctl[0] >> 4;
      console.log('cmp ' + cmp);
        switch (cmp) {
          case comp.fill:
            cli.readTightFill(rect, cb);
            break;
          case comp.png:
            cli.readTightPng(rect, cb);
            break;
          default:
            console.log('unknown tight comp!!! ' + ctl);
        }
    });
}

RfbClient.prototype.readTightFill = function(rect, cb)
{
  var stream = this.pack_stream;
  var cli = this;

  stream.get(3, function(rawbuff) {
    rect.buffer = rect.data = rawbuff;
    cli.emit('rect', rect);
    cb(rect);
  });
}

[vtolstov]

Does you mean something like this?

[sidorares]

my example data:

TIGHT RECT { x: 0, y: 0, width: 1280, height: 51, encoding: 7 }
Buffer 00 9f 57 78 9c 00 00 00 ff ff ec 5d 07 58 16 c7 d6 4e 8c 49 4c d5 68 4c 4c e2 35 f5 c6 24 6a 6e 62 2c b1 37 14 05 15 89 15 10 13 3b d6 d8 8d bd 8b 80 ... >

looks like control = 0 ( BasicCompression + no zlib ) followed by 3 bytes length = 9f 57 78 followed by len data ( which is TPIXEL fb update )

[vtolstov]

This is tight encoding, but in my case encoding -260 (tight png)

[vtolstov]

so i think that read compressed len must be exclded from png because it may be used in all tight cases...

[vtolstov]

@sidorares can you help me and extract it to dedicated function that can be used from all tight cases?

[sidorares]

looks like logic is a bit more complicated, need to implement full spec https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#tight-encoding describing all 0-7 bits of control byte to get values of stream, filter id and compression

[vtolstov]

@sidorares in case of all tight - yes, but in case of png - no. stream don't needed and filter id = 10.... but thanks for info i have more knowledge why i have issues.

[sidorares]

oh. I'm confused :)

Id Tight separate encoding to TightPng? From spec it look png is just a compression type for tight:

Bits Binary value Description 7-4 1000 FillCompression 1001 JpegCompression any other Invalid

[vtolstov]

http://wiki.qemu.org/Features/VNC_Tight_PNG

If you want I can run qemu on external IP and share it for you for easy testing

[vtolstov]

@sidorares for easy testing, please use 92.255.100.1 with port 5921 without password. If you pass via create connection encodings: [rfb.encodings.tightPng, rfb.encodings.raw] you always get tight png encoding. I'm reread spec and i think that i'm understand. When server uses TightPng it send all via tight, but when it needs to send rectangle with basic compression it uses png raw image. All other stuff like fill and filters are not used, but filter setted as i understand to png. So application know how can deal with incomping data.

[vtolstov]

@sidorares after reading fill tpixel, next rect readed fine ([224, 2] is compact length). Now i need only fix len calc, to get next rect.

[vtolstov]

@sidorares i'm complete. In my case all rects are fill or png. So last qeustion - does we need to convert fill rect on the fly to png rect or this is application level stuff? If so - how can i do that in js ?

[sidorares]

wow, great.

Not sure what api would work best. I thing we should only have very simple encodings exposed to the outside ( raw, copy rect, maybe "one solid color") or complex but well-known like png where other decoding libraries can be used. Stuff with complexity in between ( hextile, zrle etc ) should be decoded into something like raw before emitting rectangle event

[sidorares]

can you create PR with your changes? I'll test with your server tomorrow

[vtolstov]

May be the best provide native types via library and create util function to convert from native to needed type ? For example working with browser i'm prefer to convert all to png and write to canvas. But working via console, i can prefer raw... Yes, i'm create pr.

[vtolstov]

@sidorares does we need update typings file for TypeScript ?