Tidying Tictac Ecosystem for non-users

[martinsumner]

An eco-system of changes exist in Riak_KV around TictacAAE:

• the AAE process itself;
• nextgenrepl - ttaae full-sync;
• nextgenrepl - real-time repl;
• reaper;
• eraser;
• reader;
• aae fold infrastructure.

What of riak users that don't require any of these features? Some of these features create log noise (https://github.com/basho/riak_kv/pull/1875). A crashing process in one of these features can also trigger a crash of riak_kv (if done with sufficient intensity). The crashing issue has been seen when one of these processes was disabled using an environment variable change (and the disabled term of riak.conf was used instead of the boolean expected by the code).

A request has been made by @nsaadouni to look to bundle this ecosystem into a separate dependency. This might not be straight forward. Especially with respect to aae_fold support.

As a minimum though, what is proposed is:

• Have a dedicated supervisor for the Tictac EcoSystem, and start all of these processes via that supervisor, not directly via the main riak_kv supervision tree;
• Have a (default enabled) configuration option which allows the TicTac EcoSystem to be disabled (i.e. riak_kv will not start the ecosystem supervisor if the ecosystem option has been set to disabled);
• Strip where possible references to the Tictac ecosystem from riak_kv_vnode - in particular manage exchange/rebuild pokes by a ttaae manager not within the vnode itself.

For a user with a pure-basho feature set, they can be assured that their operation is not impacted by running of any processes from the nextgen ecosystem.

Another potential change might be to switch real-time repl to using a more standard mechanism to hook into the PUT process. The coordinator hook was made to reduce real-time latency, but can have issues with being too fast when not correctly configured. This would clean the riak_kv_vnode code for non-nextgenrepl users.

[fadushin]

👍🏽

Re: "real-time real" do you mean "next gen real-time repl"? (considering the fact that "legacy" realtime replication is hooked into the Put FSM already)

[martinsumner]

👍🏽

Re: "real-time real" do you mean "next gen real-time repl"? (considering the fact that "legacy" realtime replication is hooked into the Put FSM already)

Yes, that's correct. Currently nextgen real-time repl is hooked into the vnode coordinating the PUT (not as a post-commit hook), with the aim of reducing replication latency.