search

ecoppen / futuresboard

Dashboard to monitor the performance of your Binance or Bybit Futures account
http://www.futuresboard.dev/
GNU General Public License v3.0
139 stars 52 forks source link

Fix network exception #78

Closed amit-sides closed 1 year ago

amit-sides commented 1 year ago

The scraper crashed for me when it lost connection while trying to scrape. This caused it to terminate and the board didn't update until a manual restart. Here is the traceback:

raspberry@raspberrypi:~/Desktop/futuresboard $ docker-compose logs -f
Attaching to futuresboard_futuresboard_1
futuresboard_1  |  * Serving Flask app 'futuresboard.app'
futuresboard_1  |  * Debug mode: off
futuresboard_1  | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
futuresboard_1  |  * Running on all addresses (0.0.0.0)
futuresboard_1  |  * Running on http://127.0.0.1:5000
futuresboard_1  |  * Running on http://----:5000
futuresboard_1  | Press CTRL+C to quit
futuresboard_1  | [2022-11-26 11:40:42,803] INFO in scraper: Orders updated: 283; Positions updated: 19 (new: 0); Trades processed: 1811; Time elapsed: 0:01:52.883734; Sleeps: 0
futuresboard_1  | [2022-11-26 11:40:42,805] INFO in scraper: Auto scrape routines terminated. Sleeping 300 seconds...
futuresboard_1  | [2022-11-26 11:45:42,898] INFO in scraper: Auto scrape routines starting
futuresboard_1  | [2022-11-26 11:47:01,180] INFO in scraper: Orders updated: 260; Positions updated: 18 (new: 0); Trades processed: 1; Time elapsed: 0:01:18.280826; Sleeps: 0
futuresboard_1  | [2022-11-26 11:47:01,185] INFO in scraper: Auto scrape routines terminated. Sleeping 300 seconds...
futuresboard_1  | [2022-11-26 11:52:01,287] INFO in scraper: Auto scrape routines starting
futuresboard_1  | Exception in thread Thread-1:
futuresboard_1  | Traceback (most recent call last):
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
futuresboard_1  |     conn = connection.create_connection(
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 72, in create_connection
futuresboard_1  |     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
futuresboard_1  |   File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo
futuresboard_1  |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
futuresboard_1  | socket.gaierror: [Errno -3] Temporary failure in name resolution
futuresboard_1  |
futuresboard_1  | During handling of the above exception, another exception occurred:
futuresboard_1  |
futuresboard_1  | Traceback (most recent call last):
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
futuresboard_1  |     httplib_response = self._make_request(
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request
futuresboard_1  |     self._validate_conn(conn)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
futuresboard_1  |     conn.connect()
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 358, in connect
futuresboard_1  |     self.sock = conn = self._new_conn()
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
futuresboard_1  |     raise NewConnectionError(
futuresboard_1  | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fb7c48100>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution
futuresboard_1  |
futuresboard_1  | During handling of the above exception, another exception occurred:
futuresboard_1  |
futuresboard_1  | Traceback (most recent call last):
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 489, in send
futuresboard_1  |     resp = conn.urlopen(
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
futuresboard_1  |     retries = retries.increment(
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
futuresboard_1  |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
futuresboard_1  | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api-testnet.bybit.com', port=443): Max retries exceeded with url: /private/linear/position/list?api_key=----&timestamp=1669463521290&sign=---- (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fb7c48100>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
futuresboard_1  |
futuresboard_1  | During handling of the above exception, another exception occurred:
futuresboard_1  |
futuresboard_1  | Traceback (most recent call last):
futuresboard_1  |   File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
futuresboard_1  |     self.run()
futuresboard_1  |   File "/usr/local/lib/python3.8/threading.py", line 870, in run
futuresboard_1  |     self._target(*self._args, **self._kwargs)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/futuresboard/scraper.py", line 43, in _auto_scrape
futuresboard_1  |     scrape(app=app)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/futuresboard/scraper.py", line 274, in scrape
futuresboard_1  |     _scrape(app=app)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/futuresboard/scraper.py", line 405, in _scrape
futuresboard_1  |     responseHeader, responseJSON = send_signed_request(
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/futuresboard/scraper.py", line 95, in send_signed_request
futuresboard_1  |     response = dispatch_request(http_method)(**params)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 600, in get
futuresboard_1  |     return self.request("GET", url, **kwargs)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 587, in request
futuresboard_1  |     resp = self.send(prep, **send_kwargs)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
futuresboard_1  |     r = adapter.send(request, **kwargs)
futuresboard_1  |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 565, in send
futuresboard_1  |     raise ConnectionError(e, request=request)
futuresboard_1  | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api-testnet.bybit.com', port=443): Max retries exceeded with url: /private/linear/position/list?api_key=----&timestamp=1669463521290&sign=---- (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fb7c48100>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

Here is the logs after the fix:

futuresboard_1  | [2022-11-26 12:04:57,453] INFO in scraper: Auto scrape routines starting
futuresboard_1  | [2022-11-26 12:05:07,477] ERROR in scraper: Request to 'https://api-testnet.bybit.com/private/linear/position/list?api_key=-----&timestamp=1669464297456&sign=----' failed. Code: -1; Message: HTTPSConnectionPool(host='api-testnet.bybit.com', port=443): Max retries exceeded with url: /private/linear/position/list?api_key=-----&timestamp=1669464297456&sign=-----(Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7e6b58b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
futuresboard_1  | [2022-11-26 12:05:07,477] INFO in scraper: Auto scrape routines terminated. Sleeping 300 seconds...