search

criteo / babar

Profiler for large-scale distributed java applications (Spark, Scalding, MapReduce, Hive,...) on YARN.
Apache License 2.0
125 stars 29 forks source link

Trouble configuring for spark application #12

Closed frederic-schmaljohann closed 6 years ago

frederic-schmaljohann commented 6 years ago

I am having trouble configuring babar for a spark application (as described in the README).

Any ideas what could be missing? I am working with EMR on AWS with spark 2.2.1.


=======================================================================================================
LogType:stderr
Log Upload Time:Mi Jun 13 12:48:56 +0000 2018
LogLength:2725
Log Contents:
Exception thrown while starting reporter
java.io.FileNotFoundException: ./log/babar.log (No such file or directory)
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(FileOutputStream.java:270)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:101)
    at java.io.FileWriter.<init>(FileWriter.java:63)
    at com.criteo.babar.agent.reporter.LogReporter.start(LogReporter.java:28)
    at com.criteo.babar.agent.Agent.premain(Agent.java:36)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
    at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Error: Could not find or load main class JVMProfiler[reservedMB=17000]
Exception thrown while logging metric
java.lang.NullPointerException
    at com.criteo.babar.agent.reporter.LogReporter.log(LogReporter.java:59)
    at com.criteo.babar.agent.reporter.LogReporter.reportEvent(LogReporter.java:54)
    at com.criteo.babar.agent.profiler.StackTraceProfiler.report(StackTraceProfiler.java:50)
    at com.criteo.babar.agent.profiler.StackTraceProfiler.stop(StackTraceProfiler.java:67)
    at com.criteo.babar.agent.profiler.SamplingProfiler.stop(SamplingProfiler.java:47)
    at com.criteo.babar.agent.profiler.SamplingAggregatingProfiler.stop(SamplingAggregatingProfiler.java:32)
    at com.criteo.babar.agent.worker.ShutdownHookWorker.run(ShutdownHookWorker.java:23)
    at java.lang.Thread.run(Thread.java:748)