search

greglook / blocks

Clojure content-addressable data storage.
The Unlicense
113 stars 6 forks source link

Metrics wrapper for block stores #7

Closed greglook closed 7 years ago

greglook commented 8 years ago

Idea: a logical block store which is given a name and a reporting function and emits Riemann event maps to measure block store operations. These would look like:

{:time #inst "2015-11-18T19:01:45-00:00"
 :host "TBD"
 :service "block-store <store-name> <operation>"
 :metric 38.234 ; ms elapsed
 :state "ok"}

The map conversion could even be punted on, and the store could just call the function with the service name and metric. That would leave filling in the time, host, state, etc to the metrics backend.

greglook commented 7 years ago

After more thought, this is complicated by the fact that in addition to the basic BlockStore protocol, there are also the optional efficient batching and enumeration protocols. Just measuring something like block/get also doesn't capture the real IO cost of reading a block's content from the store, so this would also have to somehow wrap/instrument the input stream from the store.

Closing until I have a better way to do this. In the meantime, consumers can handle their own instrumentation.