search

hashicorp / memberlist

Golang package for gossip based membership and failure detection
Mozilla Public License 2.0
3.66k stars 440 forks source link

Fix sending compound messages corruption when including messages larger than 64KB #260

Open pracucci opened 2 years ago

pracucci commented 2 years ago

Cortex uses memberlist with a custom transport implementation, based on TCP protocol. We have a use case to send large messages, sometimes bigger than 64KB.

When memberlist implementation builds a compound message, it doesn't take in account the message length. Since in the compound message the length of an inner message is stored as a uint16, the compound message gets corrupted if contains an inner message larger than 64KB.

To solve this issue, I propose to not include messages larger than 64KB in a compound message.

Notes:

hashicorp-cla commented 2 years ago

CLA assistant check
All committers have signed the CLA.

alvinlin123 commented 2 years ago

@markan I wondering if you can take a look at this PR? I thought you might be a good candidate because you looked at the related PR: https://github.com/hashicorp/memberlist/pull/239

Thanks in advance :)

alvinlin123 commented 2 years ago

Sorry for the constant nudge, but just wondering if this PR looks good and can be merged soon?

alvinlin123 commented 2 years ago

@markan any chance you would be able to take a look at this PR soon?

alvinlin123 commented 2 years ago

No worries @markan, thank you so much for taking a look!

jeromeinsf commented 2 years ago

@pracucci Do you think you could take a look?

alvinlin123 commented 2 years ago

@Amier3 @markan I have addressed the PR review comments in separate PR: https://github.com/hashicorp/memberlist/pull/262

Please let me now if https://github.com/hashicorp/memberlist/pull/262 looks ok.