Script can crash out with "Max retries exceeded with url" from Twitter

[ara4n]

After a few(?) hours of no twitter activity, the script crashed out with requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.twitter.com', port=443): Max retries exceeded with url: /1.1/statuses/user_timeline.json?since_id=860322898879361024&tweet_mode=extended&exclude_replies=True&include_rts=False&trim_user=False (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd45fdee940>: Failed to establish a new connection: [Errno 110] Connection timed out',)).

``` $ ./MastodonToTwitter.py Tweeting any toots after toot 3560 Tooting any tweets after tweet 860322898879361024 Updated expected short URL length: Is now 24 Traceback (most recent call last): File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request self._validate_conn(conn) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 844, in _validate_conn conn.connect() File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 284, in connect conn = self._new_conn() File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) requests.packages.urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/adapters.py", line 423, in send timeout=timeout File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 649, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 376, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.twitter.com', port=443): Max retries exceeded with url: /1.1/statuses/user_timeline.json?since_id=860322898879361024&tweet_mode=extended&exclude_replies=True&include_rts=False&trim_user=False (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 110] Connection timed out',)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./MastodonToTwitter.py", line 371, in new_tweets = twitter_api.GetUserTimeline(since_id = since_tweet_id, include_rts=False, exclude_replies=True) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/twitter/api.py", line 764, in GetUserTimeline resp = self._RequestUrl(url, 'GET', data=parameters) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/twitter/api.py", line 4940, in _RequestUrl resp = requests.get(url, auth=self.__auth, timeout=self._timeout) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/api.py", line 70, in get return request('get', url, params=params, **kwargs) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/api.py", line 56, in request return session.request(method=method, url=url, **kwargs) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/sessions.py", line 488, in request resp = self.send(prep, **send_kwargs) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/home/mastodon/MastodonToTwitter/env/lib/python3.4/site-packages/requests/adapters.py", line 487, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.twitter.com', port=443): Max retries exceeded with url: /1.1/statuses/user_timeline.json?since_id=860322898879361024&tweet_mode=extended&exclude_replies=True&include_rts=False&trim_user=False (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 110] Connection timed out',)) ```

[Siltaar]

One could use this line in a crontab to automatically restart the script when it crashed :

*/2 * * * * python -c "import os;S='systemctl ';M=' MastodonToTwitter';'Active: inactive' in os.popen(S+'status'+M).read() and os.system(S+'start'+M)"

This could be interesting in all those cases : #46 #43 #42 #14 #13 …

[remram44]

The correct way is simply to have Restart=always in the service file

[ara4n]

i should have mentioned that I now run it in an 'enterprise loop' in a screen session (while true; do ./MastodonToTwitter; sleep 5; done or something), which has much the same effect. so far it hasn't broken and spammed the mastodon to death....

[Siltaar]

@remram44 Restart=always is set by default (in the /etc/systemd/system/MastodonToTwitter.service file) when installing MastodonToTwitter… yet it still needs an additional watchdog mechanism.