Add Node support - Githubissues

petethepig commented 3 years ago

TODO

[ ] research Node profiling tools

petethepig commented 3 years ago

Found this project, seems promising, but this is for heap profiling:

https://github.com/v8/sampling-heap-profiler

petethepig commented 3 years ago

~0x seems promising for CPU profiling https://github.com/davidmarkclements/0x ~

But the problem is it does the whole process from start to finish, no way to do 10 second intervals.

petethepig commented 3 years ago

This is promising https://github.com/google/pprof-nodejs

vmarchaud commented 3 years ago

@petethepig I've wrote some kind of wrapper around nodejs profiling in 2019, you can find them there: cpu and heap. Now that i re-read them i find them pretty ugly but it works quite well (i've been using in prod since).

I tried to document as much as i could back then inside both package's readme, however there are still few things worth to know:

google/pprof-nodejs is using C++ addon so it can be quite hard to setup correctly for newcomers (requires gcc when installing deps)
generally using inspector works well but you are essentialy using main thread cpu to receive/transmit profiling data back to your platform and specially goes between c++ and js which generally block userland code, hence introducing overhead.
Both of them are only able to profile JS code, some work was started to hook with pref dynamically at runtime (currently perf is only togglable with cli: docs) which would then make available profiling of C++ code too (so V8, libuv and nodejs itself)

Generally you'll find some historic in the nodejs's diag working group: https://github.com/nodejs/diagnostics/issues/148 https://github.com/nodejs/diagnostics/issues/444 https://github.com/nodejs/diagnostics/issues/150

Feel free to reach out (mail in bio) if you have any questions

Edit: Congrats on the launch on HN btw :)

vmarchaud commented 3 years ago

Also not directly related to NodeJS but you might find this interesting for browser profiling: https://wicg.github.io/js-self-profiling/

hananbeer commented 3 years ago

I don't understand why you would need to profile v8 for nodejs. that's like profiling CPython compared to simply python as you already did. let v8 devs take care of v8, they have their tools.

now for nodejs, you can hook require, this will allow you to modify any local or global module, maybe look at istanbul instrumentor on npm. you could just hook everything and only profile if delta time is over 0.01s. note that in node the architecture is event based but still a stack trace would be useful.

vmarchaud commented 3 years ago

don't understand why you would need to profile v8 for nodejs. that's like profiling CPython compared to simply python as you already did. let v8 devs take care of v8, they have their tools.

I had an issue with v8 deoptimization that pushed one of my app CPU usage to >80% (vs 40% baseline). You can also have C++ dependencies (like grpc) that won't be profiled. Even if issues like this are rare, doesn't mean its useless to support those. Altrought its hard to support perf i agree.

now for nodejs, you can hook require, this will allow you to modify any local or global module, maybe look at istanbul instrumentor on npm. you could just hook everything and only profile if delta time is over 0.01s.

I would suggest to lookup performance for this, this is highly ineficient and block the main thread (so it slow your app), the v8 profiler is using a seperate thread.

slonka commented 3 years ago

Some time ago I did a summary on NodeJS profilers: https://github.com/slonka/nodejs-production-ready-profiling - things might be outdated but you might find some useful info there.

Rperry2174 commented 2 years ago

Adding these to the list from Node.js slack:

https://github.com/Unitech/pm2 https://pm2.io/docs/enterprise/guides/profiling/ https://blog.rafaelgss.com.br/node-cpu-profiler https://github.com/serkan-ozal/middy-profiler

Rperry2174 commented 2 years ago

Added support for node!

Let us know if you have any feedback!

ben-xD commented 1 year ago

Let us know if you have any feedback!

Feedback: I don't think it's usable, which is probably why it's not a very popular package.

@Rperry2174, although Node support is added, Node 18 LTS as of 2022-10 is still not supported. pyroscope-nodejs relies on pprof, which only supports Node 10, 12 or 14.

I can see that pyroscope-nodejs is moving to use @datadog/pprof in https://github.com/grafana/pyroscope-nodejs/pull/19, but this merge request will be 1 year old next month.

It would be great to have support for Node 18 to make it usable.

grafana / pyroscope

Add Node support #8

TODO