newrelic / newrelic-jfr-core

JFR library that adapts JFR events to the New Relic Telemetry SDK
Apache License 2.0
32 stars 22 forks source link

Agent does not delete JFR recording in some cases #242

Open meiao opened 1 year ago

meiao commented 1 year ago

Description

A customer noticed that multiple recording files were in the disk and noticed the following in the agent logs.

2023-01-01T00:00:00,100+0000 [1234 1234] com.newrelic WARN: Ignoring remaining events in this file due to full queue!
2023-01-01T00:00:00,200+0000 [1234 1234] com.newrelic ERROR: Error handling raw dump file
java.util.ConcurrentModificationException: null
        at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911) ~[?:1.8.0]
        at java.util.ArrayList$Itr.next(ArrayList.java:861) ~[?:1.8.0]
        at com.newrelic.jfr.profiler.ProfileSummarizer.stackTraceToStackFrame(ProfileSummarizer.java:122) ~[newrelic.jar:8.5.0]
        at com.newrelic.jfr.profiler.ProfileSummarizer.stackTraceToEvent(ProfileSummarizer.java:102) ~[newrelic.jar:8.5.0]
        at com.newrelic.jfr.profiler.ProfileSummarizer.lambda$summarize$2(ProfileSummarizer.java:96) ~[newrelic.jar:8.5.0]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0]
        at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1723) ~[?:1.8.0]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0]
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:1.8.0]
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:1.8.0]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0]
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) ~[?:1.8.0]
        at com.newrelic.jfr.daemon.EventConverter.lambda$convert$1(EventConverter.java:80) ~[newrelic.jar:8.5.0]
        at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:1.8.0]
        at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647) ~[?:1.8.0]
        at com.newrelic.jfr.daemon.EventConverter.convert(EventConverter.java:80) ~[newrelic.jar:8.5.0]
        at com.newrelic.jfr.daemon.JFRUploader.maybeDrainAndSend(JFRUploader.java:76) ~[newrelic.jar:8.5.0]
        at com.newrelic.jfr.daemon.JFRUploader.handleFile(JFRUploader.java:43) ~[newrelic.jar:8.5.0]
        at com.newrelic.jfr.daemon.JfrController.lambda$loop$1(JfrController.java:65) ~[newrelic.jar:8.5.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0]
        at java.lang.Thread.run(Thread.java:750) [?:1.8.0]

Acceptance Criteria

The agent must always delete the JFR recording file.

workato-integration[bot] commented 1 year ago

https://issues.newrelic.com/browse/NR-170313

workato-integration[bot] commented 8 months ago

https://new-relic.atlassian.net/browse/NR-222442