jmespath / jmespath.py

JMESPath is a query language for JSON.
http://jmespath.org
MIT License
2.19k stars 181 forks source link

Update benchmark tests #104

Closed jamesls closed 8 years ago

jamesls commented 8 years ago

This includes two changes. The first is to use the benchmarks.json file from the compliance tests for running benchmarks. I made a few changes to these files I'll need to backport:

  1. Fix invalid expressions.
  2. Add results key for any expression with "bench: full". This allows for tests to still verify they get the expected results (though it's not counted against as part of the benchmark time).

The second change is to change how we run the benchmarks. The two changes here are:

  1. Run each benchmark for an allotted time budget. Now each sub test (lex, compile, search) runs for 500 ms.
  2. The results are averaged instead of using the best time. This does mean that the results will appear slightly slower than previous perf runs, but it makes more sense to use average instead of best. I also would like to add variance as well, but that's not implemented here.
codecov-io commented 8 years ago

Current coverage is 97.83%

Merging #104 into develop will not affect coverage as of 371581a

@@            develop    #104   diff @@
=======================================
  Files             8       8       
  Stmts          1018    1018       
  Branches          0       0       
  Methods           0       0       
=======================================
  Hit             996     996       
  Partial           0       0       
  Missed           22      22       

Review entire Coverage Diff as of 371581a

Powered by Codecov. Updated on successful CI builds.