Closed Dieterbe closed 8 years ago
while looking at a memprofile, I saw a surprising, and significant contribution from go-tsz.(*Series).GobDecode
go-tsz.(*Series).GobDecode
i thought we didn't do any gob decoding/encoding anymore.
(pprof) top39 -cum 2893.05MB of 3014.07MB total (95.98%) Dropped 347 nodes (cum <= 15.07MB) Showing top 39 nodes out of 50 (cum >= 38.30MB) flat flat% sum% cum cum% 0 0% 0% 3013.07MB 100% runtime.aeshashbody 0 0% 0% 2602.61MB 86.35% github.com/nsqio/go-nsq.(*Consumer).redistributeRDY 0 0% 0% 2602.61MB 86.35% main.(*Handler).HandleMessage 0 0% 0% 2466.59MB 81.84% main.(*AggMetric).Add 1348.20MB 44.73% 44.73% 2221.57MB 73.71% github.com/raintank/go-tsz.(*Series).GobDecode 188.51MB 6.25% 50.98% 1536.71MB 50.98% main.NewChunk 953.30MB 31.63% 82.61% 953.30MB 31.63% bytes.(*Buffer).grow 0 0% 82.61% 948.70MB 31.48% github.com/raintank/go-tsz.(*Iter).Next 0 0% 82.61% 945.19MB 31.36% github.com/raintank/go-tsz.(*Iter).Values 0 0% 82.61% 925.89MB 30.72% main.(*Chunk).Push 0 0% 82.61% 887.37MB 29.44% net/url.(*URL).String 0 0% 82.61% 802.34MB 26.62% net/url.(*URL).EscapedPath 0 0% 82.61% 338.54MB 11.23% main.(*DefCache).Backfill 54.51MB 1.81% 84.42% 324.04MB 10.75% github.com/raintank/raintank-metric/metricdef.IndexMetric 0 0% 84.42% 324.04MB 10.75% github.com/raintank/raintank-metric/metricdef.InitElasticsearch 0 0% 84.42% 269.53MB 8.94% encoding/binary.(*bigEndian).Uint16 0 0% 84.42% 269.53MB 8.94% encoding/binary.(*bigEndian).Uint64 0 0% 84.42% 269.53MB 8.94% encoding/json.(*decodeState).object 0 0% 84.42% 269.53MB 8.94% encoding/json.(*decodeState).unmarshal 0 0% 84.42% 125.55MB 4.17% net/url.parseHost 15.50MB 0.51% 84.94% 109.51MB 3.63% main.(*AggMetrics).GetOrCreate 109.50MB 3.63% 88.57% 109.50MB 3.63% encoding/json.(*decodeState).literalStore 94.01MB 3.12% 91.69% 94.01MB 3.12% main.NewAggMetric 0 0% 91.69% 92.53MB 3.07% reflect.(*arrayType).AssignableTo 92.53MB 3.07% 94.76% 92.53MB 3.07% reflect.mapaccess 0 0% 94.76% 61.85MB 2.05% net/http.(*ServeMux).Handle 0.50MB 0.017% 94.77% 61.35MB 2.04% main.Get 0 0% 94.77% 61.35MB 2.04% main.get.func1 0 0% 94.77% 61.35MB 2.04% net/http.(*ServeMux).match 0 0% 94.77% 61.35MB 2.04% net/http.(*conn).serve 0 0% 94.77% 54.52MB 1.81% github.com/raintank/go-tsz.New 0 0% 94.77% 41MB 1.36% encoding/json.(*decodeState).array 0 0% 94.77% 40.31MB 1.34% encoding/json.(*mapEncoder).encode 0 0% 94.77% 40.31MB 1.34% encoding/json.(*scanner).undo 0 0% 94.77% 40.31MB 1.34% encoding/json.floatEncoder.encode 0 0% 94.77% 40.31MB 1.34% encoding/json.stateRedo 0 0% 94.77% 40.31MB 1.34% encoding/json.unquoteBytes 36.50MB 1.21% 95.98% 39MB 1.29% fmt.(*buffer).WriteRune 0 0% 95.98% 38.30MB 1.27% encoding/json.(*RawMessage).UnmarshalJSON
according to the web view, both (*Chunk).Push and NewChunk() trigger this:
@Dieterbe lets update metric-tank to just use dgryski/go-tsz instead of our fork.
while looking at a memprofile, I saw a surprising, and significant contribution from
go-tsz.(*Series).GobDecode
i thought we didn't do any gob decoding/encoding anymore.
according to the web view, both (*Chunk).Push and NewChunk() trigger this: