catching ReadTimeoutError

[tropo84]

Hi,

so this might be more of an issue with me and my limited python experience, than with the actual code.

I wrote a little script to log the prices of all pairs on binance with my raspi. I am running ticker = client.get_all_tickers() every 10 seconds within a while True: loop. Everything works great, except that i get a ReadTimeOutError every once in a while (like every second day). Wether the reason is my connection or the Binance API i don't know. To solve it, i would like to catch that error, so my script can continue. My question is if the Error is actually raised to somewhere in client where i can catch it? And if so, where? Meaning what do i have to put behind except:, when doing

try:
      ticker = client.get_all_tickers()
except:  ...
      pass

Here's the error message python throws:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.5/http/client.py", line 1198, in getresponse
    response.begin()
  File "/usr/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.5/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.5/socket.py", line 576, in readinto
    return self._sock.recv_into(b)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 294, in recv_into
    raise timeout('The read operation timed out')
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 389, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 309, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.binance.com', port=443): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "CoinPriceLogger2.py", line 61, in <module>
    ticker = client.get_all_tickers()
  File "/home/pi/.local/lib/python3.5/site-packages/binance/client.py", line 381, in get_all_tickers
    return self._get('ticker/allPrices')
  File "/home/pi/.local/lib/python3.5/site-packages/binance/client.py", line 180, in _get
    return self._request_api('get', path, signed, version, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/binance/client.py", line 154, in _request_api
    return self._request(method, uri, signed, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/binance/client.py", line 148, in _request
    response = getattr(self.session, method)(uri, timeout=10, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 521, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.binance.com', port=443): Read timed out. (read timeout=10)

If anybody has a tip how to solve that, i'd really appreciate it!

[tropo84]

After digging through /binance/client.py, as well as /requests/sessions.py und /request/exceptions.py i think that the requests.exceptions are not raised into client, because in

    def _init_session(self):

        session = requests.session()
        session.headers.update({'Accept': 'application/json',
                                'User-Agent': 'binance/python',
                                'X-MBX-APIKEY': self.API_KEY})
        return session

requests.exceptions is not accessed.

So i changed it to

    def _init_session(self):

        session = requests.session()
        session.headers.update({'Accept': 'application/json',
                                'User-Agent': 'binance/python',
                                'X-MBX-APIKEY': self.API_KEY})

        session.request_exceptions=requests.exceptions        

        return session

And i think now i should be able to catch any Timeout error (either ReadTimeout or ConnectTimeout) with

try:
     ticker=client.get_all_tickers()
except: client.session.requests_exceptions.Timeout
     ...write event to logfile...
     pass

Right?

I am very new to object oriented programming so i might be wrong, but if i am indeed right (no new ReadTimeout yet happened, so i'm not sure), it would be great if this (or something similar) could be implemented into the main branch. Also it would be great to give client.session another attribute to set the timeout time to something else than 10 sec. Because right now it is hard coded by response = getattr(self.session, method)(uri, timeout=10, **kwargs) in client._request.

I changed it to response = getattr(self.session, method)(uri, timeout=self.session.timeout, **kwargs) in client._request and added session.timeout=10 in def _init_session(self):

So now i should be able to change the timeout duration by client.session.timeout=5 (for example) after calling client=client(api_key, api_sec). This should work as well, right?

[sammchardy]

Hi @tropo84 it looks like you may need to update your client which will give you the option of passing a custom timeout to the Client when you create it, or an individual call. See the docs here https://python-binance.readthedocs.io/en/latest/overview.html#requests-settings

You are correct in your case of catching the exceptions, though it should work much simpler without needing to modify the client at all.

from requests.exceptions import Timeout  # this handles ReadTimeout or ConnectTimeout

try:
     ticker=client.get_all_tickers()
except: Timeout
     # write event to logfile
     pass

[sursu]

@sammchardy to test your suggested solution, I've initiated the client as follows:

client = Client("api-key", "api-secret", {"timeout": 0.1})

The value timeout was set to was chose such that it would produce a timeout error.

Then I run:

while True:
    try:
        a = client.get_all_tickers()
    except Timeout: 
        pass

If it were to catch the Timeout error, the code would run quietly until forcefully stopped without any error, right? It does, however, spit an error and stops.

[2pd]

import time
import requests
from binance.client import Client

while True:
    time.sleep(2)
    try:
        client = Client("api-key", "api-secret", {"timeout": 0.1})
        a = client.get_all_tickers()
    except requests.exceptions.ConnectTimeout:
        print("timeout")
        pass

[sursu]

Thanks, @2pd . Replacing requests.exceptions.ConnectTimeout with requests.exceptions.ReadTimeout seems to be working for me.

[suchakpks]

client = Client("api-key", "api-secret", {"timeout": 0.1})

If I add {"timeout": 0.1} at end of parameters, it returns "SyntaxError: positional argument follows keyword argument"

How to solve it?

[MFRealG]

Hi, it didn't work for me but I made it work. When something happens with you internet connection it will continue to try to connect with delays. Script continues to work without any crashes. Does anyone knows what the "timeout" argument affects on exactly?

import sys
from binance.client import Client
import pandas as pd
import ta
from configparser import ConfigParser
from time import sleep
from binance.exceptions import BinanceAPIException, BinanceRequestException
from requests.exceptions import Timeout, ReadTimeout, ConnectTimeout, RetryError, HTTPError
from requests.exceptions import ConnectionError as CError
import socket
from urllib3.exceptions import NewConnectionError, MaxRetryError, ConnectionError
from pprint import pprint
from tabulate import tabulate

config = ConfigParser()
config.read('config.ini')
api_key = config.get('keys', 'api_key')
api_secret = config.get('keys', 'api_secret')
market = 'BTCUSDT'

while True:
    try:
        print('Trying to connect Binance Server')
        client = Client(api_key, api_secret, {"timeout": 1000}, testnet=True)
        print("Using Binance TestNet Server")
        # dft = get_market_data(market, False)
        sleep(1)
    except (BinanceAPIException, BinanceRequestException) as Binance_Error:
        print('Binance Error Exception')
        # print('Error type is', type(Binance_Error))
        print('Sleep 10 sec')
        sleep(10)
        continue
    except (socket.gaierror, NewConnectionError, MaxRetryError, ConnectionError) as Socket_Error:
        print('Socket Error Exception')
        # print('Error type is', type(Socket_Error))
        print('Sleep 10 sec')
        sleep(10)
        continue
    except (Timeout, ReadTimeout, ConnectTimeout, RetryError, HTTPError, CError) as Conn_Error:
        print('Connection Error Exception')
        # print('Error type is', type(Conn_Error))
        print('Sleep 10 sec')
        sleep(10)
        continue