search

mesosphere / marathon

Deploy and manage containers (including Docker) on top of Apache Mesos at scale.
https://mesosphere.github.io/marathon/
Apache License 2.0
4.07k stars 845 forks source link

REST api returns code 500 for invalid JSON and fails silently to proxy the error #2174

Closed mrbar42 closed 8 years ago

mrbar42 commented 9 years ago

When trying to POST an app JSON with incorrect health checks I received empty response from the api.

It took me a while to understand what's going on but I figured what's going on eventually.

when making the invalid request to a non-leader instance, the leader fails to process and the proxy request never get answered which causes this error on the non-leader instance:

[2015-09-03 12:14:57,957] INFO Proxying request to POST http://master2:8080/v2/apps from master1:8080 (mesosphere.marathon.api.JavaUrlConnectionRequestForwarder$:288)
[2015-09-03 12:15:07,985] WARN /v2/apps (org.eclipse.jetty.servlet.ServletHandler:563)
java.lang.RuntimeException: while proxying
        at mesosphere.marathon.api.LeaderProxyFilter.doFilter(LeaderProxyFilter.scala:147)
        at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
...
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:152)

When sent directly to the leader the api receives this html error message (trimmed):

<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /v2/apps. Reason:
<pre>    Server Error</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>

and from logs:

java.lang.AssertionError: assertion failed: Health check port indices must address an element of the ports array or container port mappings.
        at scala.Predef$.assert(Predef.scala:165)
        at mesosphere.marathon.state.AppDefinition.<init>(AppDefinition.scala:76)
        at mesosphere.marathon.api.v2.json.V2AppDefinition.toAppDefinition(V2AppDefinition.scala:90)
        at mesosphere.marathon.api.v2.json.V2AppDefinition.portIndicesAreValid(V2AppDefinition.scala:83)
        at mesosphere.marathon.api.v2.json.V2AppDefinition.<init>(V2AppDefinition.scala:73)
...

I think its better to reflect the result as much as possible to the API response

janisz commented 8 years ago

It looks like assertions are not necessary because there is PortIndicesValidator