jvm-profiling-tools / perf-map-agent

A java agent to generate method mappings to use with the linux `perf` tool
GNU General Public License v2.0
1.65k stars 260 forks source link

Generating mappings for transient java processes / agentlib mode #27

Open craiig opened 8 years ago

craiig commented 8 years ago

Hi all, I'm trying to profile systems that consist of lots of java processes being started and stopped over time. Does anyone know of a way to ensure that I can get a symbol mapping of each java process that gets run on my system? Just running once won't capture these transient processes, so I'd ideally like something that automatically makes a symbol dump, maybe on process termination? Any thoughts would be appreciate.

Not sure if this is the right place to ask, feel free to point me somewhere else! Thanks a lot for this tool.

jrudolph commented 8 years ago

The first question would be whether those JVM processes are running long enough to have sufficient code compiled at all. Otherwise, you'd just get a lot of opaque Interpreter frames.

The actual functionality would probably need the perf-map-agent library to be loaded at the start of each of the Java processes to capture all JIT compilation from the start of the program till the end. The problem then is that it is likely that the JVM discards some compiled methods and reuses the same memory for new methods. That means that over the run of a Java process, a memory address may map to different symbols changing over time. So, you will need a dynamic symbol mapping mechanism that associates timestamps from perf traces with timestamps from the JVM compilation events.

To get there we'd need:

All of these would be interesting to implement but not a short-term goal for now.

craiig commented 8 years ago

Thanks for the response! I'm motivated to get something like this working, since full system stack trace profilers that can examine transient JITs seem rare (or nonexistent?), but I'd like something like this for my research.

What do you think it would take to resurrect the persistent mode for the agent? I'm new to the JVMTI, but it looks like the agent would just need to implement Agent_OnLoad to register the same callbacks at startup, and just not close the the perf-map file, right? In addition to the right JAVA_TOOL_OPTIONS to get the agent loaded.

The load/unload events you're referring to are CompiledMethod(Load / Unload)?

jrudolph commented 7 years ago

Relevant new perf features:

https://lkml.org/lkml/2015/3/30/749 https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jitdump-specification.txt

libbkmz commented 10 months ago

This is old thread, but to get mappings after java process termination you can use -XX:+DumpPerfMapAtExit flag for that

jrudolph commented 10 months ago

Cool, thanks for sharing!