search

taoensso / nippy

The fastest serialization library for Clojure
https://www.taoensso.com/nippy
Eclipse Public License 1.0
1.04k stars 60 forks source link

add criterium-basd perf tests using citius #67

Closed kumarshantanu closed 9 years ago

kumarshantanu commented 9 years ago

This PR allows somebody to run comparative perf tests based on Criterium generating a bar-chart image file in the end. The PR includes tests for ["tools.reader.edn" "data.fressian" "Nippy-default" "Nippy-fast" "Nippy-encrypted" "Nippy-lzma2"]. Examples on how to run the perf tests are below:

To run all perf tests in exhaustive mode:

lein with-profile 1.7,test,perf test

To run a particular test in quick mode with vertical (non-tabular) output:

CITIUS_QUICK_BENCH=true CITIUS_CRITERIUM_OUTPUT=true lein with-profile 1.7,test,perf test :only taoensso.nippy.test-perf/test-freeze
ptaoussanis commented 9 years ago

Hi Shantanu, thanks a lot for this!

Could you get me some details on what specific need you're trying to fulfil? Is there something in particular about the current benchmark suite that you're looking to improve?

kumarshantanu commented 9 years ago

Hi Peter, the idea is to make it easy to run the benchmarks via Criterium and see the results. The current benchmark does sensible things e.g. distinguishes warmup and sample laps; Criterium's detail output would be additionally useful to look at.

ptaoussanis commented 9 years ago

Hi Shantanu,

I appreciate the work you put into this. Will be honest though, am hesitant to add (or duplicate) code that doesn't solve a pressing need.

Have been pretty short on time lately, so trying to keep the amount of code I maintain down to a minimum to help let me release more stuff. Does that make sense?

In this case, I feel like the current benchmark suite pretty much does what we need it to do (unless I'm missing something?). The results are ~accurate, in a portable map format (so easily exported to .CSV for graphing, etc.). The additional info that Criterium provides might be interesting but isn't really important to our use case here, is it?

Would of course be happy to link to your repo if you felt like maintaining it on your end?

Again, really do appreciate the time you've invested.

kumarshantanu commented 9 years ago

Makes sense, Peter.