Statistics for node local backend performance

[roder]

I think it would be good to add statistics for node local backend performance to "riak-admin status".

This would make it possible to measure effects of tuning a single node.

Something like:

node_backend_gettime_mean: <> node_backend_gettime_99: <<99% percentile for for node local gets>>

etc.

The other time measure measurements include network latencies and make it impossible to identify node local problems.

[slfritchie]

Matt, I agree, those stats are very valuable. However, it's quite possible that a node has more than one backend (via the fun of the multi backend), and it would be great for each of the "real" backends to report separate stats.

Also, the multi backend allows you to use the same "real" backend several times. It's slightly crazy, but the likeliest case for doing this is to allow a sysadmin to force bucket B1 to use the Bitcask backend that uses /path/to/B1 and bucket B2 uses the Bitcask backend that uses /some/other/way/to/B2 in order to use different file systems for the two storage areas. But because you could use Bitcask in both cases, using "bitcask" in the name of the exposed statistic should not be done.

The name that should (?) be exposed is the multi backend's description of the "real" backend, for example:

    {multi_backend_default, be_default},
    {multi_backend, [
       % format: {name, module, [Configs]}
       {be_default, riak_kv_eleveldb_backend, [
         {max_open_files, 50},
         {data_root, "/path/to/leveldb"}
       ]},
       {be_special_stuff, riak_kv_bitcask_backend, [
         {data_root, "/another/path/to/bitcask"}
      ]}
     ]},

... so then the names "be_default" and "be_special_stuff" would appear in the exposed stat names? IIRC passing those names down to the "real" backends will require a small amount of extra code.

[evanmcc]

This is important stuff, I agree, but I don't see it getting better (there is already some support for this in leveldb) for the next few releases. Marked as a feature request, closing.