WebSocket option to set max outgoing message/frame size

[gregerspoulsen]

If you exceed the websocket payload limit when issuing an RPC call in WAMP it drops the connection and fails with 'WAMP transport was lost without closing the session before'. Turning on debug logging shows that it is an 1009 error on the other end, but it took me some time to get there - I think it would be great with a more detailed main error.

That said, in my case at least, I prefer it much to detect the error on the client side to avoid dropping the connection. For now I have solved it by adding the following to autobahn.wamp.websocket.WampWebSocketProtocol.send:

if len(payload) > self.maxFramePayloadSize: raise ProtocolError(u'Message exceeds payload limit of {0}'.format(self.maxFramePayloadSize)) self.sendMessage(payload, isBinary)

maybe that would be of general interest?

[oberstet]

I prefer it much to detect the error on the client side

The problem is, WebSocket lacks a mechanism to let the other peer know maximum message/frame size limits ...

[gregerspoulsen]

Naturally it would be preferred to know the actual limits and set the outgoing limit accordingly, but when that is not an option, maybe it would make sense to make a session option with the maximum allowed outgoing message size? It could just default to the current receiver limit in the crossbar framework, and setting it to 0 could result in the current behaviour?

[oberstet]

to make a session option with the maximum allowed outgoing message size

.. and raise a client side exception already when the final, serialized message exceeds the client side set option?

That would be possible of course .. I can see that is can be more desirable than the connection being hard kicked.

[oberstet]

I have adjusted the title accordingly.

Note that there are already client side option to auto-fragment outgoing messages

[gregerspoulsen]

.. and raise a client side exception already when the final, serialized message exceeds the client side set option?

Yes, that was my thought.

Note that there are already client side option to auto-fragment outgoing messages

I am quite new to wamp and websocket, so I do not know the inner workings of auto fragmentation , but looking at the autobahn code I get the impression that this defaults to 65536, lower than the 1048576 limit, so I assume that the messages are already fragmented? or could it actually be used to circumvent the max payload size on the other end?

With regards to the outgoing check, do you want me to write it and create a pull request? And if:

• do we agree that it should be a part of ComponentConfig?
• the check should be in autobahn.wamp.websocket.WampWebSocketProtocol.send
• what would be the right way to transfer the setting to the protocol instance?

[oberstet]

WebSocket support message fragmentation (splitting a single WebSocket message into smaller frames - but with no reordering at the frame nor message level).

AutobahnPython and Crossbar.io both support the automatic fragmentation of outgoing WebSocket messages into frames.

Both also support setting a max on frame and message size of incoming messages.

What is missing is setting a max frame and/or message size of outgoing WebSocket messages.

This should be added in AB Py, and it should be added purely at the WebSocket level (not WAMP).

So the first thing we need to do is extend the setProtocolOptions with 2 new parameters:

• maxOutgoingFramePayloadSize
• maxOutgoingMessagePayloadSize

Note: auto-fragmentation of outgoing message can be controlled using autoFragmentSize parameter in above.

[gregerspoulsen]

That makes sense, I get: 1009: message exceeds payload limit of 1048576 octets. Which should appear regardless of whether auto fragmentation is enabled then.

But is your suggested maxOutgoingFramePayloadSize not the point at which it should start fragmenting the message and basically the same as autoFragmentSize then?

[oberstet]

If you set autoFragmentSize, it probably does not make sense to set maxOutgoingFramePayloadSize also.

However, the maxOutgoingMessagePayloadSize makes sense even when autoFragmentSize is set also.

[gregerspoulsen]

From your previous post I get the impression that you still think it makes sense to have maxOutgoingFramePayloadSize specified, so you can check against that when autoFragmentSize == 0. I have updated setProtocolOptions accordingly.

I guess we want to do the check in autobahn.websocket.protocol.WebSocketProtocol.sendMessage?

I am however a bit unsure what would be the right way to expose the setting. Should it be a part of ComponentConfig and handled by transport_factory.setProtocolOptions in the WAMP runner or should one manually call setProtocolOptions?

[gregerspoulsen]

Also, just to get a better understanding - how does the protocol get hold of the values from setProtocolOptions?. When I look at the code they are set as attributes of the factory, but from my previous experience with twisted the factory and protocol are separate entities, never the less the options are used from the protocol as attributes of it self?

[yli-cpr]

I just hit a similar problem when using autobahn|python as a client. In my case, the server can send large messages, but the client side runs into problem due to the maxMessagePayloadSize.

[mtrofimm]

@yli-cpr did you manage to solve the issue with large messages?

[yli-cpr]

@mtrofimm Yes. My solution is to connect to WAMP server with my own code (rather than using the ApplicationRunner), and call setProtocolOptions() on the factory object to disable maxMessagePayloadSize (=0).

[mtrofimm]

@yli-cpr Can you share some snippet how to accomplish that?

[yli-cpr]

@mtrofimm Sorry, I am working on proprietary project. But you can take a look at https://github.com/crossbario/autobahn-python/blob/2cce98a41b9ae32870f950df3072d740d2c75548/autobahn/twisted/wamp.py#L208. Write your own method based on that, remove things you do not need, and set maxMessagePayloadSize=0 at this line: https://github.com/crossbario/autobahn-python/blob/2cce98a41b9ae32870f950df3072d740d2c75548/autobahn/twisted/wamp.py#L285

[mtrofimm]

@yli-cpr thanks a lot.