Test Yokozuna and ring resize [JIRA: RIAK-1704]

[rzezeski]

Ring resize and Yokozuna have never been tested. Need to audit code for any places that do stuff with ring size or partitions. Need to make sure to update caches on ring change, etc. E.g. AAE and coverage plan. Need to devise a riak test that runs a resize while indexing and querying data.

[rzezeski]

Tests have been added for ring resize expansion (#282) but not contraction. @bowrocker and I have to talk with @jrwest and see if we can figure out why queries break on downsize.

[rzezeski]

It turns out that ring resizing and Yokozuna cannot work together at the moment without chance for indefinite incorrect results in the form of over counting.

When resizing a ring there is a chance that a) a ring index will survive (e.g. index 0) and b) it will stay on the same node. In this case the same vnode is used in the old and new ring. This vnode will send off N replicas to the new owners but it will never clean up these replicas after it is done. I.e. the vnode will contain replicas for data it no longer owns and these replicas are never cleaned up by any process.

This does not pose a problem for KV because KV will make use of the new preflist and only fetch from N vnodes. But it does pose a problem for anything that folds or uses coverage. Certain steps have been taken for things like 2i to filter this old data out on the fly as a query is being processed. See basho/riak_kv#685 for more detail.

Yokozuna cannot make similar changes as those made in KV. It uses riak core coverage to make a query plan but the actual query is executed by Solr and we have no easy way to filter old data. Fixing the incompatibility will require time and thinking which has run out for Riak 2.0. Therefore, it is my stance that these features will not work together in the 2.0 release.

[DSomogyi]

Comment for Jira.