via a PUT_FSM PUT a tombstone to replace the object;
via a GET confirm all primaries have received the tombstone, and if so send a delete request to the vnodes;
for each vnode run riak_kv_vnode:do_delete/2 which if the object is a tombstone will prompt a reap based on the delete_mode (i.e. don't reap if keep, send a timed reap if not immediate, reap straight away if immediate).
During handoffs, delete requests are not forwarded. So a delete during the handoff, will prompt the PUT of a tombstone (which is forwarded to the receiving vnode), but will only reap on the sending vnode. After a handoff, if there are deletes during the handoff, then there may be tombstones on the new (receiving) vnode - that will lead to AAE activity.
The code comment states explicitly that deletes are not forwarded, but does not explain why they cannot be forwarded. As there is a check that the object is a tombstone before reaping, it is not obviously unsafe.
martinsumner
commented
1 year ago
The basic delete process is:
riak_kv_vnode:do_delete/2which if the object is a tombstone will prompt a reap based on the delete_mode (i.e. don't reap if keep, send a timed reap if not immediate, reap straight away if immediate).During handoffs, delete requests are not forwarded. So a delete during the handoff, will prompt the PUT of a tombstone (which is forwarded to the receiving vnode), but will only reap on the sending vnode. After a handoff, if there are deletes during the handoff, then there may be tombstones on the new (receiving) vnode - that will lead to AAE activity.
The code comment states explicitly that deletes are not forwarded, but does not explain why they cannot be forwarded. As there is a check that the object is a tombstone before reaping, it is not obviously unsafe.