search

mesosphere / spark-build

Used to build the mesosphere/spark docker image and the DC/OS Spark package
52 stars 34 forks source link

Spark History Server healthcheck update from MESOS_HTTP to COMMAND #433

Closed akirillov closed 6 years ago

akirillov commented 6 years ago

Change description MESSOS_HTTP health check stopped working with Spark 2.3 History Server which binds to a private ip instead of 0.0.0.0 or 127.0.0.1. This resulted in Spark History Server healthcheck failures in Marathon which prevented it from starting.

Excerpt from the documentation, section Current Limitations:

HTTP(S) and TCP health checks use 127.0.0.1 as target IP. As a result, if tasks want to support HTTP or TCP health checks, they should listen on the loopback interface in addition to whatever interface they require (see MESOS-6517).

This patch switches from using of MESOS_HTTP to COMMAND health check which greps server logs for errors.

Related Apache Mesos issue [MESOS-6517] Health checking only on 127.0.0.1 is limiting