MA3STR0 / kimsufi-crawler

Crawler that will send you an email alert as soon as servers on OVH/Kimsufi become available for purchase
MIT License
193 stars 60 forks source link

Too many HTTP Errors: [HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=.. #61

Closed TomLewis closed 7 years ago

TomLewis commented 7 years ago

These are the errors I get after leaving the crawler running a few weeks. Not once had it send me an alert.

2016-12-18 18:04:07,957 Too many HTTP Errors: [HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330dd0>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff5e9b9d0>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff2dc8850>,request_time=0.3997790813446045,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330e30>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff5e9b990>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff2b49b90>,request_time=0.40337395668029785,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330e90>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff24befd0>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff2b49e10>,request_time=0.4039421081542969,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330ef0>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff2b681d0>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff24be710>,request_time=0.39690589904785156,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330d70>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff2333c10>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff24be610>,request_time=0.4051640033721924,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330d10>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff2b68410>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff2b495d0>,request_time=0.40108799934387207,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330cb0>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff23aaa90>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff2333bd0>,request_time=0.403148889541626,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330c50>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff23aa250>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff2b49650>,request_time=0.40256690979003906,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330bf0>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff23aa150>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff23aa8d0>,request_time=0.3977830410003662,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330b90>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff23aa710>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff23aa890>,request_time=0.4003868103027344,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330a70>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff2333f10>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff23aa6d0>,request_time=0.4134371280670166,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330a10>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff3535450>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff23aac50>,request_time=0.4054241180419922,time_info={})), HTTPError(500, 'Internal Server Error', HTTPResponse(_body=None,buffer=<_io.BytesIO object at 0x7f8ff2330ad0>,code=500,effective_url='https://ws.ovh.com/dedicated/r2/ws.dispatcher/getAvailability2',error=HTTPError(...),headers=<tornado.httputil.HTTPHeaders object at 0x7f8ff2b68e90>,reason='Internal Server Error',request=<tornado.httpclient.HTTPRequest object at 0x7f8ff23aacd0>,request_time=0.41061997413635254,time_info={}))]

MA3STR0 commented 7 years ago

Generally Error 500 indicates a server-side problem, so most likely their API was simply down. I just included better logging to catch more info in such cases.

For you a general recommendation would be to set servers to something popular like ["KS-3A", "KS-3B", "KS-2E"], which are always available, and see if notifications come through.

TomLewis commented 7 years ago

Yep! confirmed that its working with the examples you suggested, good idea! Looks like theres been sod all KS-4B's then! itching for one!