Netflix / dynomite

A generic dynamo implementation for different k-v storage engines
Apache License 2.0
4.2k stars 533 forks source link

Fix msg leak with DC_EACH_SAFE_QUORUM #721

Closed smukil closed 5 years ago

smukil commented 5 years ago

It turns out that the 'swallow' field in 'struct msg' is used in more ways than just swallowing. It also is required to basically free any cross rack and cross DC message even though we don't want to swallow it. And alternate methods are used to check if we should swallow a 'msg' or not. For example, checking in multiple places if the response is from the same DC or not and swallowing accordingly which attributes multiple purposes to the 'swallow' field and gets very confusing.

This incorrect use of 'swallow' has made it hard to create any new reasonable feature without breaking a bunch of things. This fix makes sure we don't leak msgs while using DC_EACH_SAFE_QUORUM. Even though we don't want to swallow the responses across DCs under DC_EACH_SAFE_QUORUM, we have to mark it as 'swallow = true' just to have it freed in the response path.

All of this needs to be rewritten since the codebase is basically a house of cards.