#1778 - Empty Mesos Resources Response

ngageoint / scale

Processing framework for containerized algorithms

http://ngageoint.github.io/scale/

Apache License 2.0

105 stars 45 forks source link

#1778 - Empty Mesos Resources Response #1795

Closed JohnPTobe closed 5 years ago

JohnPTobe commented 5 years ago

Checklist

[x] manage.py test passes
[x] tests are included

Affected app(s)

scheduler

Description of change

Catch exceptions encountered when getting agent resources from mesos (caused by masters restarting or anything else) and shutdown scale if exceptions continue to occur for 30 minutes. We may want to pause the scheduler and/or raise an error instead of shutting down.

JohnPTobe commented 5 years ago

I tested on the high side but scale shutdown before my code could run due to zookeeper session being lost.