Support conprof profiling - Githubissues

spacemeshos / go-spacemesh

Go Implementation of the Spacemesh protocol full node. 💾⏰💪

https://spacemesh.io

MIT License

746 stars 210 forks source link

Support conprof profiling #2641

Closed y0sher closed 2 years ago

y0sher commented 3 years ago

Description

We are currently using Pyroscope for profiling nodes in our network. Pyroscope gets resource heavy with data from a lot of node.

we want to check if conprof can provide a decent alternative and be cheaper on resources like @dshulyak mentioned.

possible issues and questions

can you collect conprof data from multiple miners ?
how much conprof instances are required ?
how much storage is required? (how does it scale with miners..?)

Requirements -

[x] - exposing http pprof for all miners
[ ] - deploying conprof and using k8s to automatically discover and add targets. (conprof works like prometheus)

dshulyak commented 3 years ago

For proof of concept we may even avoid automatio. Let's just deploy it for the current devnet to see if it will handle ~50 miners.