eprbell / dali-rp2

DaLI (Data Loader Interface) is a data loader and input generator for RP2 (https://pypi.org/project/rp2), the privacy-focused, free, open-source cryptocurrency tax calculator: DaLI removes the need to manually prepare RP2 input files. Just like RP2, DaLI is also free, open-source and it prioritizes user privacy.
https://pypi.org/project/dali-rp2/
Apache License 2.0
65 stars 42 forks source link

Raising exception when geolocked #170

Closed topherbuckley closed 1 year ago

topherbuckley commented 1 year ago

As per Issue 169

Rather than hang, this provides the following stack trace and intentionally errors out:

ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 622, in fetch
    response.raise_for_status()
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 451 Client Error:  for url: https://api.binance.com/api/v3/exchangeInfo

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/christopher/HDD/git/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 382, in find_historical_bar
    historical_data = current_exchange.fetchOHLCV(f"{from_asset}/{to_asset}", timeframe, ms_timestamp, 1)
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/binance.py", line 2522, in fetch_ohlcv
    self.load_markets()
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 1449, in load_markets
    markets = self.fetch_markets(params)
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/binance.py", line 1614, in fetch_markets
    response = getattr(self, method)(query)
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 502, in inner
    return entry(_self, **inner_kwargs)
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/binance.py", line 6042, in request
    response = self.fetch2(path, api, method, params, headers, body, config, context)
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 2802, in fetch2
    return self.fetch(request['url'], request['method'], request['headers'], request['body'])
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 640, in fetch
    self.handle_http_status_code(http_status_code, http_status_text, url, method, http_response)
  File "/media/christopher/HDD/git/dali-rp2/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 1742, in handle_http_status_code
    raise ErrorClass(self.id + ' ' + method + ' ' + url + ' ' + codeAsString + ' ' + reason + ' ' + body)
ccxt.base.errors.ExchangeNotAvailable: binance GET https://api.binance.com/api/v3/exchangeInfo 451  {
  "code": 0,
  "msg": "Service unavailable from a restricted location according to 'b. Eligibility' in https://www.binance.com/en/terms. Please contact customer service if you believe you received this message in error."
}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/media/christopher/HDD/git/dali-rp2/src/dali/dali_main.py", line 181, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
  File "/media/christopher/HDD/git/dali-rp2/src/dali/transaction_resolver.py", line 246, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
  File "/media/christopher/HDD/git/dali-rp2/src/dali/transaction_resolver.py", line 136, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
  File "/media/christopher/HDD/git/dali-rp2/src/dali/transaction_resolver.py", line 108, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
  File "/media/christopher/HDD/git/dali-rp2/src/dali/abstract_pair_converter_plugin.py", line 182, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
  File "/media/christopher/HDD/git/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 293, in get_historic_bar_from_native_source
    hop_bar = self.find_historical_bar(hop_data.from_asset, hop_data.to_asset, timestamp, hop_data.exchange)
  File "/media/christopher/HDD/git/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 395, in find_historical_bar
    raise RP2RuntimeError(f"Geolocked API when attempting to use the {exchange} exchange") from exc_na
rp2.rp2_error.RP2RuntimeError: Geolocked API when attempting to use the Binance.com exchange

I have a couple of follow up question on this though.

  1. Is this if statement necessary any longer with this PR? Was this intended to be informative for something other than this geolock exception? I was thinking to remove it, as I'm not sure how it would be informative now. It seems oddly specific to Binance, and doesn't appear to handle anything in particular compared to how it would error out in the else statement that follows.
  2. I was also thinking to add to the if statement I implemented here if "451" in str(exc_na) but I didn't like the idea of this number being possible to show up in some hash or other string of numbers. Is there any way to get the actual JSON from ccxt such that I can identify the HTTP error code as a separate entry as opposed to the compiled string from ErrorClass(self.id + ' ' + method + ' ' + url + ' ' + codeAsString + ' ' + reason + ' ' + body)?
macanudo527 commented 1 year ago

I still feel like this is too brittle. Can we close this PR for now since I'm working on a permanent fix for this?

topherbuckley commented 1 year ago

Yep, I can always reopen if I have an idea to make it more robust. Otherwise look forward to your permanent fix @macanudo527.