msgpack transportEncoding issue

[connectdotz]

hi, I am using the latest wampy.js with backend crossbar.io 0.9.12 router. Everything worked as expected when using json transportEncoding, but when I switch over to msgpack, crossbar.io complaint about ProtocolError during the initial handshaking:

...
2014-12-26 18:08:48+0000 [Router         15] RX WAMP HELLO Message (realm = test.realm.wecount, roles = [, , , ], authmethods = None, authid = None)
2014-12-26 18:08:48+0000 [Router         15] TX WAMP WELCOME Message (session = 149574515, roles = [, ], authid = yYI1wcFwlPx74I8X4Cjv06JX, authrole = anonymous, authmethod = anonymous, authprovider = None)
2014-12-26 18:08:48+0000 [Router         15] TX Frame to tcp4:192.168.59.3:51705 : fin = True, rsv = 0, opcode = 2, mask = -, length = 250, repeat_length = None, chopsize = None, sync = False, payload = 9302ce08ea537384a861757468726f6c65a9616e6f6e796d6f7573aa617574686d6574686f64a9616e6f6e796d6f7573a5726f6c657382a662726f6b657281a8666561747572657383b87075626c69736865725f6964656e74696669636174696f6ec3b37075626c69736865725f6578636c7573696f6ec3bd737562736372696265725f626c61636b77686974655f6c697374696e67c3a66465616c657281a8666561747572657382b870726f67726573736976655f63616c6c5f726573756c7473c3b563616c6c65725f6964656e74696669636174696f6ec3a6617574686964b879594931776346776c5078373449385834436a7630364a58
2014-12-26 18:08:48+0000 [Router         15] TX Octets to tcp4:192.168.59.3:51705 : sync = False, octets = 827e00fa9302ce08ea537384a861757468726f6c65a9616e6f6e796d6f7573aa617574686d6574686f64a9616e6f6e796d6f7573a5726f6c657382a662726f6b657281a8666561747572657383b87075626c69736865725f6964656e74696669636174696f6ec3b37075626c69736865725f6578636c7573696f6ec3bd737562736372696265725f626c61636b77686974655f6c697374696e67c3a66465616c657281a8666561747572657382b870726f67726573736976655f63616c6c5f726573756c7473c3b563616c6c65725f6964656e74696669636174696f6ec3a6617574686964b879594931776346776c5078373449385834436a7630364a58
2014-12-26 18:08:48+0000 [Router         15] RX Octets from tcp4:192.168.59.3:51705 : octets = 82b742bd1157d69dde5742f639e1b7525bd79bbd72382f9372382cd3743436d97e2338937c3636cf782f6cca74342dc87f236cd27f0830dc7f3c2bd376
2014-12-26 18:08:48+0000 [Router         15] RX Frame from tcp4:192.168.59.3:51705 : fin = True, rsv = 0, opcode = 2, mask = 42bd1157, length = 55, payload = 9420cf00004b28b6f5ef4a80d900636f6d2e636f6e6e656374646f747a2e6d61747269782e7765636f756e742e6f6e5f72616e6b696e67
2014-12-26 18:08:48+0000 [Router         15] Traceback (most recent call last):
2014-12-26 18:08:48+0000 [Router         15] File "/usr/local/lib/python2.7/dist-packages/autobahn/wamp/websocket.py", line 88, in onMessage
2014-12-26 18:08:48+0000 [Router         15] for msg in self._serializer.unserialize(payload, isBinary):
2014-12-26 18:08:48+0000 [Router         15] File "/usr/local/lib/python2.7/dist-packages/autobahn/wamp/serializer.py", line 106, in unserialize
2014-12-26 18:08:48+0000 [Router         15] raise ProtocolError("invalid serialization of WAMP message ({0})".format(e))
2014-12-26 18:08:48+0000 [Router         15] ProtocolError: invalid serialization of WAMP message (unpack(b) received extra data.)
2014-12-26 18:08:48+0000 [Router         15] Failing WAMP-over-WebSocket transport: code = 1002, reason = 'WAMP Protocol Error (invalid serialization of WAMP message (unpack(b) received extra data.))'
201

there is no warning on the wampy.js side. The failed message seems to be from the subscribe() call. I am not sure if it is a crossbar.io or or wampy.js issue, but giving crossbar.io is one of the most popular/mature WAMP router implementation, I would think it is important to make sure wampy.js is compatible.

Please let me know if there is anything I can provide to help track down this issue. Thanks.

[KSDaemon]

Hi, @connectdotz! First of all, do you use msgpack.js that is included with wampy? Also, please post example of your js code for connecting to crossbar using wampy with msgpack encoding.

[connectdotz]

Hi,

First of all, do you use msgpack.js that is included with wampy?

Yes

please post example of your js code for connecting to crossbar using wampy with msgpack encoding

    var connection = new Wampy(service_options.router,
        { realm: service_options.realm }
    );
    connection.options({
        reconnectInterval: 10000,
        maxRetries: 20,
        transportEncoding: 'msgpack', // this is the only difference between json and msgpack encoding
        onConnect: function () { 
            ...
            connection.subscribe(uri, {
                onSuccess: function () { console.log('Successfully subscribed to ' + uri); },
                onError: function (err) { console.log('Failed to subscribe to ' + uri + ', error:' + err); },
                onEvent: function (results) { 
                    console.log('Received ranking result: ' + results); 
                }
            });
            console.log('Yahoo! We are online!'); 
        },
        onClose: function () { 
            console.log('See you next time!'); 
        },
        onError: function () { 
            console.log('Breakdown happened'); 
        },
        onReconnect: function () { 
            console.log('Reconnecting...'); 
        }
    });
    connection.connect();

[KSDaemon]

okey! I'll try to check wampy with crossbar and wiola.

[connectdotz]

One more info, here is the config file on the crossbar side:

{
   "controller": {
   },
   "workers": [
      {
         "type": "router",
         "options": {
            "pythonpath": [".."]
         },
         "realms": [
            {
               "name": "test.realm.x",
               "roles": [
                  {
                     "id": "anonymous",
                     "name": "anonymous",
                     "permissions": [
                        {
                           "uri": "*",
                           "publish": true,
                           "subscribe": true,
                           "call": true,
                           "register": true
                        }
                     ]
                  }
               ]
            }
         ],
         "transports": [
            {
               "type": "websocket",
               "endpoint": {
                  "type": "tcp",
                  "port": 8080
               },
               "debug": true
            }
         ]
      }
   ]
}

[KSDaemon]

Well, i tried to remember, what was the problem, cause i was investigating that thing in past... Thankfully, goolge remember all things :) so you can read more about this problem in this thread: https://groups.google.com/forum/#!searchin/wampws/msgpack/wampws/eDfKK2agYjo/gcjnqs9jYGMJ New year is comming, and i'm going on holidays, if i'd have time, i'll try to investigate deeper. But may be you also can help.

[connectdotz]

Happy New Year then.

Where did you stand regarding the thread you mentioned above? Were you ever able to get msgpack protocol working with crossbar.io or any other routers?

[KSDaemon]

Hi! Thanks! Fuh, Holidays are over! (We have holidays from 30.12 till 11.01). Well, what can i say:

• Yes, i know a working msgpack connectivity between wampy.js and wiola WAMP router (another mine WAMP project)
• As i've investigated, problem is on python side, cause its msgpack serializer encode strings as binary array (0xc4), but it should be strings ((type & 0xe0) === 0xa0)

I have a some kind of comparement file, you can download it here: http://ksdaemon.ru/dl/msgpack-test.zip Just open html file. There is a table where javascript msgpack lib is used to decode messages, encoded by itself, python lib and lua lib. From there you can see the difference in encoding strings.

[KSDaemon]

What is most interesting, is that python encoded messages are very good decoded by lua, and there are no problems there. But when javascript is decoding python messages, there are problems. I can fix that by interpreting bin8 as strings, but that is not well.

[KSDaemon]

Can not reproduce this bug in current versions.

"msgpack5": "^3.4.1"

Crossbar.io        : 17.2.1 (Crossbar.io COMMUNITY)
   Autobahn         : 0.17.2 (with JSON, MessagePack, CBOR, UBJSON)
   Twisted          : 17.1.0-EPollReactor
   LMDB             : 0.92/lmdb-0.9.18
   Python           : 2.7.12/CPython

So, closing.