search

qedus / osmpbf

OpenStreetMap PBF file format parser in Go Lang.
MIT License
141 stars 29 forks source link

Use benchstat instead of benchcmp. #27

Closed AlekSi closed 6 years ago

AlekSi commented 6 years ago

A side-effect of me benchmarking various projects I contributed to with the upcoming Go 1.10.

$ benchstat go1.9.2_01.txt goDevel_01.txt

name                old time/op    new time/op    delta
Decode-4               2.27s ± 3%     2.16s ± 2%  -5.10%  (p=0.008 n=5+5)
DecodeConcurrent-4     2.30s ± 1%     2.08s ± 3%  -9.80%  (p=0.008 n=5+5)

name                old alloc/op   new alloc/op   delta
Decode-4              1.89GB ± 0%    1.88GB ± 0%  -0.11%  (p=0.008 n=5+5)
DecodeConcurrent-4    1.89GB ± 0%    1.88GB ± 0%  -0.11%  (p=0.008 n=5+5)

name                old allocs/op  new allocs/op  delta
Decode-4               13.0M ± 0%     13.0M ± 0%  +0.00%  (p=0.008 n=5+5)
DecodeConcurrent-4     13.0M ± 0%     13.0M ± 0%  +0.00%  (p=0.008 n=5+5)
$ benchstat go1.9.2_32.txt goDevel_32.txt

name                old time/op    new time/op    delta
Decode-4               1.53s ± 2%     1.44s ± 1%  -5.94%  (p=0.008 n=5+5)
DecodeConcurrent-4     1.61s ± 3%     1.48s ± 1%  -7.72%  (p=0.008 n=5+5)

name                old alloc/op   new alloc/op   delta
Decode-4              1.92GB ± 0%    1.92GB ± 0%  -0.10%  (p=0.008 n=5+5)
DecodeConcurrent-4    1.92GB ± 0%    1.92GB ± 0%  -0.10%  (p=0.008 n=5+5)

name                old allocs/op  new allocs/op  delta
Decode-4               13.0M ± 0%     13.0M ± 0%  +0.00%  (p=0.008 n=5+5)
DecodeConcurrent-4     13.0M ± 0%     13.0M ± 0%  +0.00%  (p=0.008 n=5+5)
AlekSi commented 6 years ago

benchstat requires Go 1.6+ for text/template "trim space" ({{-) feature. Are we ok to drop some old version of Go from Travis CI? What should we keep?

jongillham commented 6 years ago

I'm happy with dropping Go 1.6+.