Kraken Rest API _initialize_markets broken

[JasonSimms]

I just saw the new merge and wanted to give it a go. Seems to break on kraken.py

File "/home/jason/.local/lib/python3.8/site-packages/dali/plugin/input/rest/kraken.py", line 135, in _initialize_markets self.base_id_to_base.update({market[_BASE_ID]: market[_BASE]}) TypeError: list indices must be integers or slices, not str.

Also thanks for the work of this implemenation. I've gotta get this done this week! Yikes. ----

LOG --- The log debug print is the market variable

INFO: Reading crypto data using plugin 'dali.plugin.input.rest.kraken' DEBUG initialize markets:> debug the market in markets... [{'id': '1INCHEUR', 'symbol': '1INCH/EUR', 'base': '1INCH', 'quote': 'EUR', 'settle': None, 'baseId': '1INCH', 'quoteId': 'ZEUR', 'settleId': None, 'darkpool': False, 'altname': '1INCHEUR', 'type': 'spot', 'spot': True, 'margin': False, 'swap': False, 'future': False, 'option': False, 'active': True, 'contract': False, 'linear': None, 'inverse': None, 'taker': 0.0026, 'maker': 0.0016, 'contractSize': None, 'expiry': None, 'expiryDatetime': None, 'strike': None, 'optionType': None, 'precision': {'amount': 1e-08, 'price': 0.001}, 'limits': {'leverage': {'min': 1.0, 'max': 1}, 'amount': {'min': 10.0, 'max': None}, 'price': {'min': 0.001, 'max': None}, 'cost': {'min': None, 'max': None}}, 'info': {'altname': '1INCHEUR', 'wsname': '1INCH/EUR', 'aclass_base': 'currency', 'base': '1INCH', 'aclass_quote': 'currency', 'quote': 'ZEUR', 'lot': 'unit', 'cost_decimals': '5', 'pair_decimals': '3', 'lot_decimals': '8', 'lot_multiplier': '1', 'leverage_buy': [], 'leverage_sell': [], 'fees': [['0', '0.26'], ['50000', '0.24'], ['100000', '0.22'], ['250000', '0.2'], ['500000', '0.18'], ['1000000', '0.16'], ['2500000', '0.14'], ['5000000', '0.12'], ['10000000', '0.1']], 'fees_maker': [['0', '0.16'], ['50000', '0.14'], ['100000', '0.12'], ['250000', '0.1'], ['500000', '0.08'], ['1000000', '0.06'], ['2500000', '0.04'], ['5000000', '0.02'], ['10000000', '0.0']], 'fee_volume_currency': 'ZUSD', 'margin_call': '80', 'margin_stop': '40', 'ordermin': '10', 'costmin': '0.45', 'tick_size': '0.001', 'status': 'online'}}] ERROR: Fatal exception occurred: Traceback (most recent call last): File "/home/jason/.local/lib/python3.8/site-packages/dali/dali_main.py", line 176, in _dali_main_internal result_list = pool.map(_input_plugin_helper, input_plugin_args_list) File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get raise self._value File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, *kwds)) File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar return list(map(args)) File "/home/jason/.local/lib/python3.8/site-packages/dali/dali_main.py", line 214, in _input_plugin_helper plugin_transactions = input_plugin.load(country) File "/home/jason/.local/lib/python3.8/site-packages/dali/plugin/input/rest/kraken.py", line 190, in load self._initialize_markets() File "/home/jason/.local/lib/python3.8/site-packages/dali/plugin/input/rest/kraken.py", line 135, in _initialize_markets self.base_id_to_base.update({market[_BASE_ID]: market[_BASE]}) TypeError: list indices must be integers or slices, not str INFO: Log file: ./log/rp2_2023_05_15_12_21_01_768621.log INFO: Generated output directory: output

[JasonSimms]

Im not sure what the BVUSD line is doing it doesn't seem to fit the format from line 109. The api is returning an array so I changed the lines to reference the location of baseID and base. Excuse my dev ignorance but i wasnt confident it still fulfills the same functions anymore.

self.base_id_to_base: Dict[str, str] = {} format KRAKEN.py

def _initialize_markets(self) -> None:
        self._client.load_markets()
        self._client.markets_by_id.update({"BSVUSD": {_ID: "BSVUSD", _BASE_ID: "BSV", _BASE: "BSV", _QUOTE: "USD"}}) #What is this?
        for market in self._client.markets_by_id.values():
            if isinstance(market, list):
                self.base_id_to_base.update({market[0][_BASE_ID]: market[0]["info"][_BASE]})  #API is returning a list 
            else:
                print("this market is not a list")
                print(type(market))
                print(market)

[eprbell]

@ndopencode, would you be able to take a look at this?

[ndopencode]

ah yes... we were seeing this before. In CCXT, the attribute for the class Exchange, markets_by_id, is typically a dictionary where the market IDs are used as keys to access market information. However, the error, TypeError: list indices must be integers or slices, not str, suggests that it is trying to use a string value as an index for a list. I have to take a little bit of time to understand why this is happening.

@JasonSimms, when you ran this what was the command you used at the command line?

[alexeckert]

Stumbled upon the same issue, as mentioned above the values of markets_by_id are actually a list of len 1 which contain a dict.

[eprbell]

Calling for people with a Kraken account to help out on this and submit a PR. Labeled "Help Wanted".

[ndopencode]

@eprbell @macanudo527 @JasonSimms @alexeckert I didn't mean to delay this issue, I've been away busy with personal business. I think I've solved the problem. It had to do with differing ccxt versions in my environment. I had an older ccxt that was passing back a Dict but the newer `ccxt' changed this to return a List. I've created a PR to fix this issue.

Pull Request can be found here: https://github.com/eprbell/dali-rp2/pull/198/

[eprbell]

@eprbell @macanudo527 @JasonSimms @alexeckert I didn't mean to delay this issue, I've been away busy with personal business. I think I've solved the problem. It had to do with differing ccxt versions in my environment. I had an older ccxt that was passing back a Dict but the newer `ccxt' changed this to return a List. I've created a PR to fix this issue.

Pull Request can be found here: #198

Welcome back! PR reviewed.