Open victornoel opened 6 months ago
I was able to get the same issue. All of my logs are filling with this error :/
Does the error happen with the async-profiler https://github.com/async-profiler/async-profiler ?
Please specify exact java versions (including vendor, build number) or base docker image
Hi @korniltsev, in my case, it is using Java 21 with the base docker image eclipse-temurin:21-jre-jammy
.
I don't remember ever seeing this error when using async-profiler, but it's hard to know as I mostly used it on my local machine (no docker). I will try to find some time to test it out, but I have to admit I can't guarantee I will be able to do it soon :P
Hi @korniltsev, today I tested running some java in the exact same setup as when using pyroscope but without pyroscope and only async-profiler v3.0 added as an agent to the VM arguments and there are no warnings.
Since pyroscope loads the agent at runtime, maybe this is where the difference is… also I'm not sure if pyroscope uses the latest version of async-profiler (3.0)? this could also be an explanation…
we are experiencing the same issue with eclipse-temurin:21.0.2_13-jdk
and agent v0.14.0
facing the same behaviour even with 0.13.1 javaagent and following env variables:
PYROSCOPE_APPLICATION_NAME=app;
PYROSCOPE_LABELS=server=XXXX;
PYROSCOPE_LOG_LEVEL=debug;
PYROSCOPE_PROFILER_ALLOC=512k;
PYROSCOPE_PROFILER_EVENT=itimer;
PYROSCOPE_PROFILER_LOCK=1ms;
PYROSCOPE_PROFILING_INTERVAL=100ms;
PYROSCOPE_UPLOAD_INTERVAL=1s;
PYROSCOPE_SERVER_ADDRESS=url
If I remove PYROSCOPE_PROFILER_LOCK
, I don't see warnings anymore
updates:
profiler start - profiler stop - dump - start - stop - dump ...
https://github.com/grafana/pyroscope-java/blob/c79e21a2beb1f76080e0184f0c8a8d4e49b92fec/agent/src/main/java/io/pyroscope/javaagent/impl/SamplingProfilingScheduler.java#L48-L96so we have multiple times when profiler do class/traps registration and it's create a lot of WARN messages from jvm
I also don't know is it correct logic or not to register/de-register async-profiler multiple times
workaround -Xlog:jni+resolve=off
maybe make sense to validate logic how pyroscope use async-profiler and do run only once and extract dump for specific interval, instead of multiple start/stop?
maybe make sense to validate logic how pyroscope use async-profiler and do run only once and extract dump for specific interval, instead of multiple start/stop?
absolutely, pyroscope should not load asprof twice. I will take a look, this should not be the case.
met too. Seems it will continuous cost many CPU times on JIT compile.
Describe the bug
When Pyroscope agent is loaded at runtime on a Java 21 VM, I can see the following warnings every few seconds in the logs:
To Reproduce
The agent is loaded with:
Expected behavior
No warnings
Environment
Additional Context
I found this old JDK bug meant to be fixed in Java 15: https://bugs.openjdk.org/browse/JDK-8239593