douban / pymesos

A pure python implementation of Mesos scheduler and executor
BSD 3-Clause "New" or "Revised" License
163 stars 88 forks source link

Follow 307 redirect for Mesos operator #112

Closed irvinlim closed 6 years ago

irvinlim commented 6 years ago

I was getting errors like this when a new Mesos master was elected as leader and the old Mesos master was return 307 status codes, as described here:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-97ca5d57dd56> in <module>()
----> 1 mesos_manager.master_operator.getAgents()

/usr/local/lib/python2.7/dist-packages/pymesos/operator_v1.pyc in getAgents(self)
    261             type='GET_AGENTS',
    262         )
--> 263         return self._send(body)
    264
    265     def getRoles(self):

/usr/local/lib/python2.7/dist-packages/pymesos/operator_v1.pyc in _send(self, body, path, method, headers)
     62                 raise RuntimeError(
     63                     'Failed to send request code=%s, message=%s' % (
---> 64                         resp.status, resp.read()
     65                     )
     66                 )

RuntimeError: Failed to send request code=307, message=

I have added a check for 3xx response codes, and updated the daemon URI accordingly for subsequent requests.

I'm not sure if this is also required for scheduler.py, as I found that my scheduler worker was still working fine despite the leader change, not sure why as well.

On the other hand, should I be using MasterDetector to handle this case instead? In any case, I feel like we should be supporting both use cases for Mesos master HA in any case.

ariesdevil commented 6 years ago

As to your question, I think the scheduler uses the underlying process.py to detect mesos master change event, see: https://github.com/douban/pymesos/blob/master/pymesos/process.py#L145

ariesdevil commented 6 years ago

LGTM. Thanks for your PR

irvinlim commented 6 years ago

Thank you! Sorry for the multiple commits, forgot that it should be on Python 3 as well. Can I get you to publish a version ASAP? Thank you! 🙇

ariesdevil commented 6 years ago

@irvinlim Done. And multiple commits are OK, case Github support squash multi-commits.