search

segmentio / stats

Go package for abstracting stats collection
https://godoc.org/github.com/segmentio/stats
MIT License
208 stars 32 forks source link

better GC stats #66

Closed achille-roussel closed 7 years ago

achille-roussel commented 7 years ago

I noticed GC stats seemed inflated in some cases, the logic for stripping out outdated GC stats seemed hard to debug and error prone, so I replaced it to use the values in the runtime.MemStats structure.

This also makes the GC stats collection more efficient as it doesn't require calling debug.ReadGCStats anymore.

I based most of this work on the runtime.MemStats documentation.

Here's a trace of the test showing GC stats computation:

=== RUN   TestGoMetrics
--- PASS: TestGoMetrics (0.11s)
    go_test.go:19: collect number 0
    go_test.go:27: { go.runtime(gauge:cpu.num=4, gauge:goroutine.num=2, counter:cgo.calls=1) [version=go1.9] }
    go_test.go:27: { go.memstats(gauge:alloc.bytes=332560, counter:total_alloc.bytes=332560, counter:lookups.count=3, counter:mallocs.count=2718, counter:frees.count=253) [type=total, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:alloc.bytes=332560, gauge:sys.bytes=1769472, gauge:idle.bytes=909312, gauge:inuse.bytes=860160, counter:released.bytes=0, gauge:objects.count=2465) [type=heap, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=327680, gauge:sys.bytes=327680) [type=stack, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=13528, gauge:sys.bytes=16384) [type=mspan, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=6944, gauge:sys.bytes=16384) [type=mcache, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=2345) [type=bucket_hash_table, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=137216) [type=gc, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=814807) [type=other, version=go1.9] }
    go_test.go:27: { go.memstats(counter:gc.count=0, gauge:gc_next.bytes=4473924, gauge:gc_pause.seconds.avg=0s, gauge:gc_pause.seconds.min=0s, gauge:gc_pause.seconds.max=0s, gauge:gc_cpu.fraction=0) [version=go1.9] }
    go_test.go:19: collect number 1
    go_test.go:27: { go.runtime(gauge:cpu.num=4, gauge:goroutine.num=2, counter:cgo.calls=0) [version=go1.9] }
    go_test.go:27: { go.memstats(gauge:alloc.bytes=87528, counter:total_alloc.bytes=33120, counter:lookups.count=0, counter:mallocs.count=573, counter:frees.count=2649) [type=total, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:alloc.bytes=87528, gauge:sys.bytes=1736704, gauge:idle.bytes=1228800, gauge:inuse.bytes=507904, counter:released.bytes=0, gauge:objects.count=389) [type=heap, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=360448, gauge:sys.bytes=360448) [type=stack, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=10336, gauge:sys.bytes=16384) [type=mspan, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=6944, gauge:sys.bytes=16384) [type=mcache, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=2345) [type=bucket_hash_table, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=274432) [type=gc, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=808663) [type=other, version=go1.9] }
    go_test.go:27: { go.memstats(counter:gc.count=1, gauge:gc_next.bytes=4194304, gauge:gc_pause.seconds.avg=52.508µs, gauge:gc_pause.seconds.min=52.508µs, gauge:gc_pause.seconds.max=52.508µs, gauge:gc_cpu.fraction=-3.678665601751283e-11) [version=go1.9] }
    go_test.go:19: collect number 2
    go_test.go:27: { go.runtime(gauge:cpu.num=4, gauge:goroutine.num=2, counter:cgo.calls=0) [version=go1.9] }
    go_test.go:27: { go.memstats(gauge:alloc.bytes=89160, counter:total_alloc.bytes=28312, counter:lookups.count=0, counter:mallocs.count=263, counter:frees.count=265) [type=total, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:alloc.bytes=89160, gauge:sys.bytes=1736704, gauge:idle.bytes=1220608, gauge:inuse.bytes=516096, counter:released.bytes=0, gauge:objects.count=387) [type=heap, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=360448, gauge:sys.bytes=360448) [type=stack, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=10488, gauge:sys.bytes=16384) [type=mspan, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:inuse.bytes=6944, gauge:sys.bytes=16384) [type=mcache, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=2345) [type=bucket_hash_table, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=274432) [type=gc, version=go1.9] }
    go_test.go:27: { go.memstats(gauge:sys.bytes=808663) [type=other, version=go1.9] }
    go_test.go:27: { go.memstats(counter:gc.count=2, gauge:gc_next.bytes=4194304, gauge:gc_pause.seconds.avg=12.01µs, gauge:gc_pause.seconds.min=11.511µs, gauge:gc_pause.seconds.max=12.509µs, gauge:gc_cpu.fraction=-7.030487196218036e-11) [version=go1.9] }

Please take a look and let me know if something should be changed.