Crash "ConnectionError: BadStatusLine"

[kristoffer-tvera]

I have a very simple script. It makes an unverified account, sleeps 5 minutes, then does a get_posts_recent(skip=0, limit=60, after="", mine=False, hashtag="", channel="")

Pretty much taken straight out of the documentation/samples. PS I also tried the version without brackets, get_post_recent() (got the idea from your test file, its in one of the first tests).

However, a random number of cycles (usually 1-5), it will crash, with the error stack:

Traceback (most recent call last): File "C:\PythonScript\jodler.py", line 31, in <module> looped_recent = j.get_posts_recent(skip=0, limit=60, after="", mine=False, hashtag="", channel="") File "C:\Python27\lib\site-packages\jodel_api.py", line 214, in get_posts_recent return self._get_posts('', skip, limit, after, mine, hashtag, channel, **kwargs) File "C:\Python27\lib\site-packages\jodel_api.py", line 211, in _get_posts return self._send_request("GET", url, params=params, **kwargs) File "C:\Python27\lib\site-packages\jodel_api.py", line 62, in _send_request resp = s.request(method=method, url=url, params=params, json=payload, headers=headers, **kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request resp = self.send(prep, **send_kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send r = adapter.send(request, **kwargs) File "C:\Python27\lib\site-packages\requests\adapters.py", line 426, in send raise ConnectionError(err, request=request) ConnectionError: ('Connection aborted.', BadStatusLine("''",))

Line 31 in my script is as follows:

looped_recent = j.get_posts_recent(skip=0, limit=60, after="", mine=False, hashtag="", channel="")

Nothing fancy going on, but it dies. Any idea?

[kristoffer-tvera]

Here is pip -freeze: cffi==1.5.0 cryptography==1.2.1 Django==1.9.5 enum34==1.1.2 funcsigs==1.0.2 future==0.16.0 idna==2.0 image==1.5.3 ipaddress==1.0.16 jodel-api==1.1.3 mock==2.0.0 ndg-httpsclient==0.4.0 oauthlib==2.0.2 pbr==3.0.0 Pillow==3.2.0 pyasn1==0.1.9 PyAutoGUI==0.9.33 pycparser==2.14 pydel==1.0.0 PyMsgBox==1.0.3 pyOpenSSL==0.15.1 PyScreeze==0.1.8 python-dateutil==2.4.2 PyTweening==1.0.3 requests==2.9.0 requests-oauthlib==0.8.0 simplejson==3.8.1 six==1.10.0 tqdm==3.7.1 tweepy==3.5.0 twitter==1.17.1 virtualenv==14.0.6

[kristoffer-tvera]

I dont know if it helps, but here is your test suit ran:

`===Flaky Test Report===

test_reinitalize passed 1 out of the required 1 times. Success! test_refresh_access_token passed 1 out of the required 1 times. Success! test_set_location passed 1 out of the required 1 times. Success! test_read_posts_recent passed 1 out of the required 1 times. Success! test_get_my_posts passed 1 out of the required 1 times. Success! test_newsfeed_after failed (1 runs remaining out of 2). <type 'exceptions.AssertionError'> assert 403 == 200 [<TracebackEntry C:\test_jodel_api.py:69>] test_newsfeed_after failed; it passed 0 out of the required 1 times. <type 'exceptions.AssertionError'> assert 403 == 200 [<TracebackEntry C:\test_jodel_api.py:69>] test_popular_after passed 1 out of the required 1 times. Success! test_get_posts_channel passed 1 out of the required 1 times. Success! test_get_channels passed 1 out of the required 1 times. Success! test_follow_channel passed 1 out of the required 1 times. Success! test_get_config passed 1 out of the required 1 times. Success! test_notifications passed 1 out of the required 1 times. Success! test_captcha passed 1 out of the required 1 times. Success! test_verify_success passed 1 out of the required 1 times. Success! test_verify_fail passed 1 out of the required 1 times. Success! test_post_details passed 1 out of the required 1 times. Success! test_post_details_v3 passed 1 out of the required 1 times. Success! test_share_url passed 1 out of the required 1 times. Success! test_pin passed 1 out of the required 1 times. Success! test_vote passed 1 out of the required 1 times. Success! test_switch_notifications passed 1 out of the required 1 times. Success! test_bad_gateway_retry passed 1 out of the required 1 times. Success! test_bad_gateway_no_retry passed 1 out of the required 1 times. Success!`

[nborrmann]

That error means that you're receiving a malformed response or the server prematurely terminated the connection (cf [1], [2]).

It's weird that you are getting the error so often, I can't remember ever seeing this. It might be due to your specific combination of your OS, python version, ssl lib, even your ISP. Try different machines/connections and see if you're still getting the error.

If nothing works, I'd recommend generously wrapping this in a try except and retrying upon an error.

Anyway, there's not much jodel_api can do about that, so I'm closing this issue.

[1] https://github.com/kennethreitz/requests/issues/2364

[2] https://bugs.python.org/issue23054

[nborrmann]

Are you maybe behind a proxy that might close the connection?

So we end up raising a BadStatusLine exception for a completely permissible case: the server closed a persistent connection. This causes persistent connections to fail for users of httplib in mysterious ways, especially if they are behind proxies: Squid, for example, seems to limit persistent connections to a maximum of 3 requests, and then closes the connection, causing future requests to raise the BadStatusLine.

This error has apparently been fixed in python 3.5.

[kristoffer-tvera]

Thanks for getting back to me so quick during a weekend.

I ran into same issues in 2.7 and 3.5. I am not behind a proxy, but i haven't tested the code on another connection, so it might come down to the connection.

I sat this up as a text-to-speech bot about 4 months ago, and didn't recall running into any issues back then, so I assumed the API handled these network issues.

Wrapping everything up in try/catch loops works. I also bumped every instance of time.sleep up to double, and have been able to cycle through 72 times now without catching a single exception. Strange.