search

uber-common / jvm-profiler

JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter
Other
1.79k stars 342 forks source link

Run JVM Profiler on a regular Filesystem #41

Closed ramshankar1990 closed 5 years ago

ramshankar1990 commented 5 years ago

I am trying to use the JVM profiler on a supercomputing cluster. To run the tool it requires the HDFS URL but the admins who are helping me with setting up the tool on the cluster would like to know if this tool can be run on a regular filesystem since using the HDFS URL could open up security issues from their end. I appreciate your help in this regard.

hiboyang commented 5 years ago

Hi Ram, yes, you could use it on a regular file system, with following steps:

  1. Copy jvm-profiler-1.0.0.jar file to each machine in your cluster.
  2. Add following argument to your java command: -javaagent:path/to/jvm-profiler-1.0.0.jar=reporter=com.uber.profiling.reporters.ConsoleOutputReporter
ramshankar1990 commented 5 years ago

Thank you for the quick response. I will update the admins about this. I am trying to profile a Spark job using the tool. May I know after copying the jar on to the cluster would the command to profile the Spark job be the same as shown in the repo?