search

goncalotomas / FMKe

🛠️ Realistic benchmark for key value stores
Other
23 stars 8 forks source link

implement experimental populate interface #213

Closed goncalotomas closed 5 years ago

goncalotomas commented 5 years ago

In normal conditions, the database population step happens through the fmke and driver interfaces. This allows us to populate any supported database, but has some performance implications that are significant when running batches of experiments in cloud systems. I implemented a new fmke_populate module that supports some of the slowest to populate databases (e.g. riak takes about 45 minutes to load the full FMKe dataset) where insert operations are performed blindly, without previous reads or writes. This should in theory give us a good throughput increase during the initial population stage for these supported databases.

The fmke_populator tool checks at start whether the fmke_populate module is available and if the database is supported. The default operation mode that uses the main fmke module is chosen if either of these checks fail.

coveralls commented 5 years ago

Coverage Status

Coverage decreased (-4.07%) to 85.563% when pulling 5cfeeb27d3d38ddcf6b7055612c3c712b60efb6a on feature/populate-interface into 855a0e4bf7e11fa430e1e65e641eb42299223b02 on master.