search

uber-common / jvm-profiler

JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter
Other
1.78k stars 342 forks source link

Using fileappender as output option #49

Closed ajames-branch closed 5 years ago

ajames-branch commented 5 years ago

While profiling a spark application, if we are using the following setting: reporter=com.uber.profiling.reporters.FileOutputReporter,outputDir=/tmp/ProfilerOut, is the output present in all the executor nodes or will it get aggregated and driver writes on its local directory?

ajames-branch commented 5 years ago

Also, do we have any metrics on the latency difference when we execute the application with and without profiling? Trying to understand if adding the profiler will add lag to the application execution.

felixcheung commented 5 years ago

I think FileOutputReporter will be written to local directories. about latency or overhead, @boy-uber should have the number to share, but it should be very low.

ajames-branch commented 5 years ago

Thank you @felixcheung