search

amqp / rhea

A reactive messaging library based on the AMQP protocol
Apache License 2.0
273 stars 80 forks source link

Combine transfer frames after all frames have been received #382

Closed kohtala closed 2 years ago

kohtala commented 2 years ago

Large message reception performance is bad due to repeated calls to Buffer.concat to grow the buffer (#381).

This addresses that at the transfer frame reception so that smaller max_frame_size helps avoid large Buffer allocations. There are still repeated smaller Buffer allocations reading the frame.

kohtala commented 2 years ago

I also updated the tests for new node versions. Seems 17.x has some problems.

This also carries fix to split outgoing messages to respect max_frame_size and an untested change to send as many frames as fits in the window for a message larger than the window.

kohtala commented 2 years ago

The node 17x. test needs new keys and certificates for upgraded OpenSSL in node 17. I'm short on time to find again how those were made.

kohtala commented 2 years ago

The large frame reception is now fast enough to be usable. I added a method to transport to peek if the size can be known from the first bytes.

I did take a look at delaying fragmenting the message on send, but did not see a simple solution. I think there may be something funny with the Outgoing.window and that would need to be studied at the same time. If I understood right, it should know beforehand the number of transfer frames needed for the pending deliveries. Currently it seems to just count down from types.MAX_UINT.

Node 18 had become current. I changed the versions to test.

grs commented 2 years ago

Thanks again @kohtala !