Large frames with Spring based brokers

[nohanna]

I'm having an issue since my update from Angular 5 / ng2-stompjs 4.0.0 to Angular 7 / ng2-stompjs 7.0.1. It seems to be related to the content length of my file that I'm sending.

I'm actually sending a text file using the publish method. I've 2 files (which are converted into a string before sending with publish) : the first one is ~14.3kB and the other one is ~14kB - something like 13.980kB. The file with 14kB works fine, it is sent correctly and received by my server without any issue. But the second file with 14.3kB never reach my server and my stomp connection is lost and then reconnects instantly. In that case, sometimes I receive this error :

Error : WebSocket connection to 'ws://localhost:4200/' failed: One or more reserved bits are on: reserved1 = 1, reserved2 = 0, reserved3 = 0

or this one :

WebSocket connection to 'ws://localhost:4200/ws-test' failed: Invalid frame header

When I check my frames, I can see in the data tab SEND destination:my_destination [...] and in the length tab 16770. And just after this, another frame (Opcode -1).

I think it is not due to my server because I changed the max length in my configuration, and it worked fine with the 4.0.0 of ng2-stompjs version. I don't think it is related to the Angular version too, so that's why I'm posting here.

Do you remember changing something in the code related to this issue?

[kum-deepak]

Actually the code related to this functionality has been rewritten bottom up.

If the file is not confidential in nature please attach the file here or email me at deepak@kreatio.com

Which broker are you using?

[nohanna]

I can't share the file unfortunately. It is a simple text file that I'm sending.

I'm using pure WebSocket, I tried to use SockJS and it's not working at all (even for the 14kB file). Actually I'm using Angular 7 / RxStomp with WebSocket (no SockJS).

[kum-deepak]

The test cases cover 1MB payloads. So, not sure what may be the issue.

The test cases are run against RabbitMQ using plain WebSockets.

[nohanna]

This is really weird. I will try to figure it out, maybe I missed something. If I find a solution, I will post it here if it's related to STOMP.

Is the StompRService still available in the version 7 of ng2-stomp? I will try to use StompRService instead of RxStompService to see if there is any difference.

[kum-deepak]

StompRService is still available but it uses RxStompService under the hood. Still it may be worth a try.

One thing that has changed in handling of text content - it does expect strings to be valid UTF8 now. You can use try using binaryBody option (https://stomp-js.github.io/api-docs/latest/classes/RxStomp.html#publish). When using binaryBody, it does not try to verify or convert the data.

See also: https://stomp-js.github.io/guide/stompjs/2018/06/28/using-stompjs-v5.html#binary-messages

[nohanna]

I don't think my files are not UTF8 valid because both files are exactly identical. Actually, my file which is never sent is the exact same of the one which can be sent with just 1 line removed. I can remove any line in the file, it will work.

I tried with binaryBody and tried to use StompRService too, I've the same problem. It really seems to be related to the file length.

[kum-deepak]

I am assuming you are able to receive large body payload but are not able to send it.

I have gone through the differences one more time. One more aspect has changed, for larger data size it used to send as split packets (which is not strictly complaint to the protocol).

Which broker are you using? It will help me in replicating the issue.

[nohanna]

Yes, I'm still investigating on that. I'm using the default spring websocket message broker.

[kum-deepak]

That probably is one of the brokers (actually only one that I know of) that supports receiving split packets the way previous version of @stomp/stompjs did.

I will not be able to test this feature, on my own as I do not have required Spring/Java knowledge. However, if you are able to share the broker side code, with instruction to run, I will attempt to reintroduce the older splitting behavior as an option.

[nohanna]

Ok, I will create a case that reproduces my configuration. I will do it as soon as I can (2-3 days) because I've not enough time right now.

Thanks for the help.

[kum-deepak]

Good news that I have been able to send the larger packet with added code for splitting large packets. However this is not compliant with WebSocket and STOMP specifications. Actually this mode causes large packets to fail with compliant brokers.

So, it will need to added as an optional flag. Since the change is in the lowest library, it will involve releasing all the three libraries.

I have started work on @stomp/stompjs - https://github.com/stomp-js/stompjs/commit/8ad4485b53c101b06446680f58d15d97c74d90bd

Need to add test cases there, then I will move onto @stomp/rx-stomp and ng2-stompjs.

[nohanna]

Really good news indeed ! So it should only be required for Spring brokers then.

Thanks for your work !

[kum-deepak]

Please check 7.2.0-beta.1 release. Set splitLargeFrames to true in options. See if it works.

[nohanna]

It works well for me with the splitLargeFrames option. I tried with a 283kB file and received the correct file in my server.

[kum-deepak]

Great, I will make a public release of 7.2.0.

[nohanna]

Very good ! Thanks a lot :)

[kum-deepak]

Released.

[jvogtherr]

I am experiencing the following problem: When sending a large message, the ws connection closes and then reconnects with the message disappearing as a result. The message however is not as large as i would expect it to have to be for the connection to stop, it says content-length: 17502. For smaller messages everything works fine. Server-side, the maximum content-length is much much higher. What is going on? I appreciate any input, thank you very much.

[nohanna]

If you are using a Spring broker, you have to add splitLargeFrames: true in your Stomp configuration. Large frames causes issues with Spring broker.

[jvogtherr]

Thank you for your quick reply. Where do I have to add this, though? StompConfig in my NgModule file seems to be the wrong place

[nohanna]

When you call your RxStompService.configure(my_config) you can pass a InjectableRxStompConfig before calling RxStompService.activate() where you can define options like heartbeats, reconnectDelay and splitLargeFrames. Everything is detailed here: https://stomp-js.github.io/api-docs/latest/classes/RxStompConfig.html.

Here is an example of a full Stomp configuration: https://github.com/nohanna/stomp-issue/blob/master/stomp-test/angularFrontend/src/app/services/stomp.service.ts

[kum-deepak]

There is a general guide at https://stomp-js.github.io/guide/ng2-stompjs/2018/11/04/ng2-stomp-with-angular7.html and a sample at https://github.com/stomp-js/ng2-stompjs-angular7.

In addition with Spring you would need to set splitLargeFrames option.

[jvogtherr]

These examples use RxStompConfig, however i use the basic StompConfig. Does it work the same way? Should i migrate to RxStompConfig?

[nohanna]

For newer version, you should migrate to RxStompConfig. As @kum-deepak said

StompRService is still available but it uses RxStompService under the hood.

But I think you should migrate to RxStompService (so RxStompConfig too) to stay aligned with future releases.

[jvogtherr]

Thank you very much