moderato-app / live-pprof

Monitor Go app's performance in seconds, not a day.
282 stars 5 forks source link

cpu profiling already in use #26

Open eccles opened 1 month ago

eccles commented 1 month ago

Thought I might give some feedback.

I have managed to get a pprof server running in one pod in my k8s cluster and have forwarded a port for one pod.

Running live-pprof seems to work. There are repeated Internal Server Error Could not enable CPU profiling - already in use. This was with the pod not doing anything. Once I started running tests this warning went away.

However the graphs do not show anything like the examples shown in the README.

Screenshot from 2024-09-11 16-04-44

eccles commented 1 month ago

nice idea - this has saved me having to setup special collectors and grafana dashboards

clement2026 commented 1 month ago

Thanks for the feedback! I'm glad you found it useful.

However the graphs do not show anything like the examples shown in the README.

Under the hood, live-pprof fetches CPU usage from http://{host}:{port}/debug/pprof/profile?seconds={seconds} and uses the pprof library to parse the response. The debug/pprof/profile endpoint is locked and can only be accessed by one client at a time. If another client tries to access it during the seconds duration, it gets the error: Could not enable CPU profiling: cpu profiling already in use.

It's easy to trigger the error when someone opens two pages if both start monitoring.

The error also happens if someone opens the URL editor while the page monitoring is on, because they both try to visit the /debug/pprof/profile endpoint:

image

These are expected errors.

However, the error also occurs when the webpage code has bugs, like misusing React hooks, which is unexpected. Refreshing the page usually fixes this. But if the error keeps showing up, then we need to look into it.


However the graphs do not show anything like the examples shown in the README.

I run into this problem sometimes too. Usually, it’s because my target app doesn’t use much CPU. pprof skips nodes that use less than 10 ns of CPU. Since live-pprof just shows what pprofgives, sometimes there’s nothing to display in the charts.

Xnip2024-09-12_07-47-04

This might be what pprof shows if my app uses just a bit of CPU:

image
Oloruntobi1 commented 1 month ago

I've hit the same problem too. One way i tried to avoid it was only running the live-pprof http://localhost:6060/debug/pprof command instead of live-pprof 6060 command.

Next time i trigger it, i'll watch for what i did.