WikiWatershed / mmw-geoprocessing

A Spark Job Server job for Model My Watershed geoprocessing.
Apache License 2.0
6 stars 6 forks source link

Collections API: Improve Error Reporting #69

Closed rajadain closed 6 years ago

rajadain commented 6 years ago

Currently, when there is an error, the output is very minimal:

$ http :8090/run < nlcd-kfactor-mismatch-request.json

HTTP/1.1 500 Internal Server Error
Content-Length: 35
Content-Type: text/plain; charset=UTF-8
Date: Wed, 30 Aug 2017 15:56:44 GMT
Server: akka-http/10.0.9

There was an internal server error.

While the log is pretty verbose:

[ERROR] [08/30/2017 15:56:44.158] [WebServer-akka.actor.default-dispatcher-62] [akka.actor.ActorSystemImpl(WebServer)] Error during processing of request: 'key not found: SpatialKey(277,82)'. Completing with 500 Internal Server Error response. To change default exception handling behavior, provide a custom ExceptionHandler.
java.util.NoSuchElementException: key not found: SpatialKey(277,82)
    at scala.collection.MapLike$class.default(MapLike.scala:228)
    at scala.collection.AbstractMap.default(Map.scala:59)
    at scala.collection.MapLike$class.apply(MapLike.scala:141)
    at scala.collection.AbstractMap.apply(Map.scala:59)
    at org.wikiwatershed.mmw.geoprocessing.Utils$$anonfun$joinCollectionLayers$2$$anonfun$6.apply(Utils.scala:151)
    at org.wikiwatershed.mmw.geoprocessing.Utils$$anonfun$joinCollectionLayers$2$$anonfun$6.apply(Utils.scala:151)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.immutable.List.map(List.scala:296)
    at org.wikiwatershed.mmw.geoprocessing.Utils$$anonfun$joinCollectionLayers$2.apply(Utils.scala:151)
    at org.wikiwatershed.mmw.geoprocessing.Utils$$anonfun$joinCollectionLayers$2.apply(Utils.scala:150)
    at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:683)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:682)
    at org.wikiwatershed.mmw.geoprocessing.Utils$class.joinCollectionLayers(Utils.scala:150)
    at org.wikiwatershed.mmw.geoprocessing.WebServer$.joinCollectionLayers(WebServer.scala:34)
    at org.wikiwatershed.mmw.geoprocessing.Geoprocessing$class.rasterGroupedAverage(Geoprocessing.scala:175)
    at org.wikiwatershed.mmw.geoprocessing.Geoprocessing$class.getRasterGroupedAverage(Geoprocessing.scala:45)
    at org.wikiwatershed.mmw.geoprocessing.WebServer$.getRasterGroupedAverage(WebServer.scala:34)
    at org.wikiwatershed.mmw.geoprocessing.WebServer$$anonfun$routes$2$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$6.apply(WebServer.scala:50)
    at org.wikiwatershed.mmw.geoprocessing.WebServer$$anonfun$routes$2$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$6.apply(WebServer.scala:50)
    at akka.http.scaladsl.server.directives.RouteDirectives$$anonfun$complete$1.apply(RouteDirectives.scala:47)
    at akka.http.scaladsl.server.directives.RouteDirectives$$anonfun$complete$1.apply(RouteDirectives.scala:47)
    at akka.http.scaladsl.server.StandardRoute$$anon$1.apply(StandardRoute.scala:19)
    at akka.http.scaladsl.server.StandardRoute$$anon$1.apply(StandardRoute.scala:19)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRouteResult$1$$anonfun$apply$3.apply(BasicDirectives.scala:61)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRouteResult$1$$anonfun$apply$3.apply(BasicDirectives.scala:61)
    at akka.http.scaladsl.server.directives.FutureDirectives$$anonfun$onComplete$1$$anonfun$apply$1$$anonfun$apply$2.apply(FutureDirectives.scala:37)
    at akka.http.scaladsl.server.directives.FutureDirectives$$anonfun$onComplete$1$$anonfun$apply$1$$anonfun$apply$2.apply(FutureDirectives.scala:37)
    at akka.http.scaladsl.util.FastFuture$$anonfun$transformWith$extension0$1.apply(FastFuture.scala:37)
    at akka.http.scaladsl.util.FastFuture$$anonfun$transformWith$extension0$1.apply(FastFuture.scala:37)
    at akka.http.scaladsl.util.FastFuture$.akka$http$scaladsl$util$FastFuture$$strictTransform$1(FastFuture.scala:41)
    at akka.http.scaladsl.util.FastFuture$.transformWith$extension1(FastFuture.scala:45)
    at akka.http.scaladsl.util.FastFuture$.transformWith$extension0(FastFuture.scala:37)
    at akka.http.scaladsl.server.directives.FutureDirectives$$anonfun$onComplete$1$$anonfun$apply$1.apply(FutureDirectives.scala:37)
    at akka.http.scaladsl.server.directives.FutureDirectives$$anonfun$onComplete$1$$anonfun$apply$1.apply(FutureDirectives.scala:35)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRequestContext$1$$anonfun$apply$1.apply(BasicDirectives.scala:43)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRequestContext$1$$anonfun$apply$1.apply(BasicDirectives.scala:43)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRouteResult$1$$anonfun$apply$3.apply(BasicDirectives.scala:61)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRouteResult$1$$anonfun$apply$3.apply(BasicDirectives.scala:61)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.RouteConcatenation$RouteWithConcatenation$$anonfun$$tilde$1$$anonfun$apply$1.apply(RouteConcatenation.scala:47)
    at akka.http.scaladsl.server.RouteConcatenation$RouteWithConcatenation$$anonfun$$tilde$1$$anonfun$apply$1.apply(RouteConcatenation.scala:44)
    at akka.http.scaladsl.util.FastFuture$.akka$http$scaladsl$util$FastFuture$$strictTransform$1(FastFuture.scala:41)
    at akka.http.scaladsl.util.FastFuture$.transformWith$extension1(FastFuture.scala:45)
    at akka.http.scaladsl.util.FastFuture$.flatMap$extension(FastFuture.scala:26)
    at akka.http.scaladsl.server.RouteConcatenation$RouteWithConcatenation$$anonfun$$tilde$1.apply(RouteConcatenation.scala:44)
    at akka.http.scaladsl.server.RouteConcatenation$RouteWithConcatenation$$anonfun$$tilde$1.apply(RouteConcatenation.scala:42)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRouteResultWith$1$$anonfun$apply$4.apply(BasicDirectives.scala:67)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$mapRouteResultWith$1$$anonfun$apply$4.apply(BasicDirectives.scala:67)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.BasicDirectives$$anonfun$textract$1$$anonfun$apply$6.apply(BasicDirectives.scala:154)
    at akka.http.scaladsl.server.directives.ExecutionDirectives$$anonfun$handleExceptions$1$$anonfun$apply$1.apply(ExecutionDirectives.scala:32)
    at akka.http.scaladsl.server.directives.ExecutionDirectives$$anonfun$handleExceptions$1$$anonfun$apply$1.apply(ExecutionDirectives.scala:28)
    at akka.http.scaladsl.server.Route$$anonfun$asyncHandler$1.apply(Route.scala:79)
    at akka.http.scaladsl.server.Route$$anonfun$asyncHandler$1.apply(Route.scala:78)
    at akka.stream.impl.fusing.MapAsync$$anon$24.onPush(Ops.scala:1169)
    at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:747)
    at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:710)
    at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:616)
    at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:471)
    at akka.stream.impl.fusing.GraphInterpreterShell.receive(ActorGraphInterpreter.scala:423)
    at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:603)
    at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:618)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
    at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:529)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
    at akka.actor.ActorCell.invoke(ActorCell.scala:495)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

It would be useful to return the first line of the error in the response:

java.util.NoSuchElementException: key not found: SpatialKey(277,82)

This example uses nlcd-kfactor-mismatch-request.json.txt as a sample request.

kellyi commented 6 years ago

For reference: http://doc.akka.io/docs/akka-http/current/scala/http/routing-dsl/exception-handling.html