search

hashicorp / nomad-spark

DEPRECATED: Apache Spark with native support for Nomad as a scheduler
44 stars 16 forks source link

Executor page does not display on 2.2.0 #7

Closed alexnaspo closed 5 years ago

alexnaspo commented 6 years ago

When running a spark streaming application on nomad 2.2.0 the executor page does not display. When running the same application on 2.1.0, it displays as expected.

Upon inspecting the network calls, it seems the <service>/api/v1/applications returns the following exception.

HTTP ERROR 500
Problem accessing /api/v1/applications. Reason:

    Server Error
Caused by:
java.lang.NullPointerException
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
    at org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
    at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
    at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:461)
    at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
    at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.spark_project.jetty.server.Server.handle(Server.java:524)
    at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:319)
    at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
    at org.spark_project.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.spark_project.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.spark_project.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.spark_project.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:748)
barnardb commented 6 years ago

I'm unable to reproduce your issue. It looks a lot like SPARK-22365, which seems to be similar to SPARK-15854. I suspect you have something on your classpath that conflicts with a Spark dependency. Are there any stacktraces earlier in your log?

In particular, as suggested by the stacktrace shown in SPARK-15854, it you might have something on the classpath that is providing a competing implementation of the javax.ws.rs.core.Application class that doesn't have the expected getProperties() method. Spark expects to get this class from:

      <dependency>
        <groupId>javax.ws.rs</groupId>
        <artifactId>javax.ws.rs-api</artifactId>
        <version>2.0.1</version>
      </dependency>

For one commenter on SPARK-22365 it was a jsr311-api-1.1.1.jar, but there seem to be many other maven artifacts containing a class with that name. The end of this blog post seems to show a similar conflict trying to run the history server with a Jersey jar on the classpath.

I hope this helps.