search

diffeo / kvlayer

python abstraction for key-value databases with key-prefix scans that supports Accumulo, HBase, Postgres, Mysql, Redis, and more. Can switch backends with just a configuration change. E.g., develop on Postgres and then switch to HBase (nosql) without changing code.
8 stars 1 forks source link

Move test_throughput_insert_random to a script #3

Open diegows opened 10 years ago

diegows commented 10 years ago

Create an utility outside of the tests to check the throughput of kvlayer, agaisnt Accumulo at first.

This script should read the current cluster configuration (address, credentials) from a file in the local directory (config_accumulo.yaml). This file will be generated by hand at first but then will be created/updated by the "make cluster" target.

Use this script to tests Accumulo and to discover bugs in kvlayer an in the DB.

johnrfrank commented 10 years ago

This utility should capture both aggregate throughput across multiple workers, and also the latency experienced by each worker. The latency statistics should be recorded, and the output should summarize min, 5th percentile, median, mean, 95th percentile, and max latency.

This script should be added as an entry point and included in the egg, so that anyone using kvlayer can easily check their database system's throughput/latency while under load. Here is an example of adding an entry point