search

eprbell / dali-rp2

DaLI (Data Loader Interface) is a data loader and input generator for RP2 (https://pypi.org/project/rp2), the privacy-focused, free, open-source cryptocurrency tax calculator: DaLI removes the need to manually prepare RP2 input files. Just like RP2, DaLI is also free, open-source and it prioritizes user privacy.
https://pypi.org/project/dali-rp2/
Apache License 2.0
65 stars 42 forks source link

Add aliases #196

Closed macanudo527 closed 1 year ago

macanudo527 commented 1 year ago

This addresses #140

This will need to be tested by users that need to use aliases in their data - @kiates, @raizazel, @Larision

Larision commented 1 year ago 
2023-07-18 15:34:49,392/dali/ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/kali/renta/dali-rp2maca/src/dali/dali_main.py", line 184, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/transaction_resolver.py", line 138, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/transaction_resolver.py", line 110, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/abstract_pair_converter_plugin.py", line 254, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/ccxt.py", line 278, in get_historic_bar_from_native_source
    self._cache_graph_snapshots(exchange)
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/ccxt.py", line 654, in _cache_graph_snapshots
    optimizations = self._optimize_assets_for_exchange(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/ccxt.py", line 762, in _optimize_assets_for_exchange
    bar_check = self.find_historical_bars(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/ccxt.py", line 441, in find_historical_bars
    csv_bar = csv_reader.find_historical_bars(from_asset, to_asset, timestamp, True, _ONE_WEEK)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/csv/kraken.py", line 352, in find_historical_bars
    if self._download_and_chunk(base_asset, quote_asset, all_bars):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/csv/kraken.py", line 361, in _download_and_chunk
    file_bytes = self._google_file_to_bytes(base_file)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/renta/dali-rp2maca/src/dali/plugin/pair_converter/csv/kraken.py", line 494, in _google_file_to_bytes
    with ZipFile(BytesIO(file_response.content)) as file_check:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/zipfile.py", line 1302, in __init__
    self._RealGetContents()
  File "/usr/lib/python3.11/zipfile.py", line 1369, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

im getting this error when running dali-rp2 with your changes.

macanudo527 commented 1 year ago

That looks like an error with the Kraken CSV plugin caused by a corrupted zipfile. The download probably got interrupted or some other issue. That admittedly needs to be more robust. I don't think it has anything to do with aliases though.

Larision commented 1 year ago

seems, not working for me. aliases write any line in logs? i didnt see any.

2023-07-20 20:31:02,651/dali/DEBUG: Self-contained transaction: InTransaction:
  plugin=Coinbase
  unique_id=xxx
  raw_data={"id": "db9cd6ef-xxx", "type": "staking_reward", "status": "completed", "amount": {"amount": "0.00000296", "currency": "ETH2"}, "native_amount": {"amount": "0.00", "currency": "EUR"}, "description": null, "created_at": "2023-07-04T04:23:04Z", "updated_at": "2023-07-04T04:23:04Z", "resource": "transaction", "resource_path": "/v2/accounts/x-x-x-x-x/transactions/x-x-x-x-x", "instant_exchange": false, "from": {"id": "x-x-x-x-x", "resource": "user", "resource_path": "/v2/users/x-x-x-x-x", "currency": "ETH2"}, "details": {"title": "Recompensa de Ethereum 2", "subtitle": "De Coinbase", "header": "Se han recibido 0,00000296 ETH2 (0,00 \u20ac)", "health": "positive"}, "hide_native_amount": false}
  timestamp=2023-07-04 04:23:04+0000
  asset=ETH2
  exchange=Coinbase
  holder=fran
  transaction_type=Staking
  spot_price=0.91923957636
  crypto_in=0.00000296
  fiat_fee=0
  fiat_in_no_fee=None
  fiat_in_with_fee=None
  notes=nearest spot_price read from CCXT-converter plugin; Recompensa de Ethereum 2
2023-07-20 20:31:02,651/CCXT-converter/nearest/DEBUG: Converting ETH2 to EUR
2023-07-20 20:31:02,651/CCXT-converter/nearest/DEBUG: Using default exchange Gate type for Coinbase.
2023-07-20 20:31:02,662/CCXT-converter/nearest/DEBUG: Found path - ['ETH2', 'USDT', 'USD', 'EUR']
2023-07-20 20:31:02,662/dali/DEBUG: Checking if ETH2 is in {'LBP', 'ETB', 'PYG', 'JEP', 'RSD', 'BOB', 'FORTH', 'XAG', 'QAR', 'DZD', 'SHP', 'FET', 'NZD', 'ILS', 'EGP', 'JOD', 'CDF', 'KMF', 'MRU', 'UMA', 'GEL', 'MRO', 'MVR', 'MATIC', 'NGN', 'CZK', 'KES', 'VND', 'XAU', 'UYU', 'EUR', 'LSL', 'HUF', 'BGN', 'NMR', 'RLY', 'ZAR', 'PHP', 'XAF', 'XLM', 'ETH', 'BHD', 'KRW', 'IOTX', 'COMP', 'HTG', 'SDG', 'BZD', 'STD', 'IMP', 'KGS', ' ETH2', 'CNH', 'DOP', 'OMR', 'COP', 'SVC', 'RWF', 'CTX', 'GGP', 'ANKR', 'BOND', 'CUC', 'GNF', 'KHR', 'MKD', 'LTC', 'NEAR', 'NU', 'CUP', 'IRR', 'SZL', 'UGX', 'TMT', 'MUR', 'TWD', 'XDR', 'ACH', 'GTQ', 'BDT', 'NIO', 'AMP', 'LAK', 'SCR', 'MAD', 'BIF', 'SOS', 'XCD', 'STN', 'JMD', 'HOPR', 'CHZ', 'CLV', 'SAND', 'RON', 'ALGO', 'BTC', 'ERN', 'MNT', 'KZT', 'BND', 'MWK', 'GIP', 'BWP', 'MMK', 'THB', 'XPD', 'USDT', 'JPY', 'SSP', 'BRL', 'XPT', 'ZWL', 'RUB', 'SLL', 'ETH2', 'PEN', 'FKP', 'SEK', 'TOP', 'GMD', 'NPR', 'GAL', 'GYD', 'SHIB', 'BAL', 'ATOM', 'XBT', 'ALEPH', 'CVE', 'SAR', 'QNT', 'NAD', 'PKR', 'XOF', 'VES', 'PLN', 'GBP', 'GHS', 'RNDR', 'SYP', 'TJS', 'XCN', 'CLF', 'INR', 'PAB', 'LYD', 'RLC', 'NOK', 'KPW', 'XPF', 'SBD', 'LUNA', 'IDR', 'MYR', 'CAD', 'ACS', 'DKK', 'TND', 'AUCTION', 'LRD', 'VUV', 'HKD', 'MZN', 'DJF', 'TZS', 'BSD', 'VEF', 'IQD', 'TRY', 'KAVA', 'UZS', 'BTN', 'USD', 'BYN', 'MOP', 'BMD', 'KYD', 'ADA', 'CRC', 'ZMW', 'MGA', 'SKL', 'YER', 'CHF', 'SRD', 'CNY', 'HNL', 'GRT', 'KWD', 'ISK', 'CLP', 'UNI', 'UAH', 'CTSI', 'FJD', 'HRK', 'LKR', 'SGD', 'PGK', 'USDC', 'MXN', 'WST', 'MDL', 'LRC', 'TTD'}
2023-07-20 20:31:02,662/dali/DEBUG: Checking if USDT is in {'LBP', 'ETB', 'PYG', 'JEP', 'RSD', 'BOB', 'FORTH', 'XAG', 'QAR', 'DZD', 'SHP', 'FET', 'NZD', 'ILS', 'EGP', 'JOD', 'CDF', 'KMF', 'MRU', 'UMA', 'GEL', 'MRO', 'MVR', 'MATIC', 'NGN', 'CZK', 'KES', 'VND', 'XAU', 'UYU', 'EUR', 'LSL', 'HUF', 'BGN', 'NMR', 'RLY', 'ZAR', 'PHP', 'XAF', 'XLM', 'ETH', 'BHD', 'KRW', 'IOTX', 'COMP', 'HTG', 'SDG', 'BZD', 'STD', 'IMP', 'KGS', ' ETH2', 'CNH', 'DOP', 'OMR', 'COP', 'SVC', 'RWF', 'CTX', 'GGP', 'ANKR', 'BOND', 'CUC', 'GNF', 'KHR', 'MKD', 'LTC', 'NEAR', 'NU', 'CUP', 'IRR', 'SZL', 'UGX', 'TMT', 'MUR', 'TWD', 'XDR', 'ACH', 'GTQ', 'BDT', 'NIO', 'AMP', 'LAK', 'SCR', 'MAD', 'BIF', 'SOS', 'XCD', 'STN', 'JMD', 'HOPR', 'CHZ', 'CLV', 'SAND', 'RON', 'ALGO', 'BTC', 'ERN', 'MNT', 'KZT', 'BND', 'MWK', 'GIP', 'BWP', 'MMK', 'THB', 'XPD', 'USDT', 'JPY', 'SSP', 'BRL', 'XPT', 'ZWL', 'RUB', 'SLL', 'ETH2', 'PEN', 'FKP', 'SEK', 'TOP', 'GMD', 'NPR', 'GAL', 'GYD', 'SHIB', 'BAL', 'ATOM', 'XBT', 'ALEPH', 'CVE', 'SAR', 'QNT', 'NAD', 'PKR', 'XOF', 'VES', 'PLN', 'GBP', 'GHS', 'RNDR', 'SYP', 'TJS', 'XCN', 'CLF', 'INR', 'PAB', 'LYD', 'RLC', 'NOK', 'KPW', 'XPF', 'SBD', 'LUNA', 'IDR', 'MYR', 'CAD', 'ACS', 'DKK', 'TND', 'AUCTION', 'LRD', 'VUV', 'HKD', 'MZN', 'DJF', 'TZS', 'BSD', 'VEF', 'IQD', 'TRY', 'KAVA', 'UZS', 'BTN', 'USD', 'BYN', 'MOP', 'BMD', 'KYD', 'ADA', 'CRC', 'ZMW', 'MGA', 'SKL', 'YER', 'CHF', 'SRD', 'CNY', 'HNL', 'GRT', 'KWD', 'ISK', 'CLP', 'UNI', 'UAH', 'CTSI', 'FJD', 'HRK', 'LKR', 'SGD', 'PGK', 'USDC', 'MXN', 'WST', 'MDL', 'LRC', 'TTD'}
2023-07-20 20:31:02,662/dali/DEBUG: Checking if USD is in {'LBP', 'ETB', 'PYG', 'JEP', 'RSD', 'BOB', 'FORTH', 'XAG', 'QAR', 'DZD', 'SHP', 'FET', 'NZD', 'ILS', 'EGP', 'JOD', 'CDF', 'KMF', 'MRU', 'UMA', 'GEL', 'MRO', 'MVR', 'MATIC', 'NGN', 'CZK', 'KES', 'VND', 'XAU', 'UYU', 'EUR', 'LSL', 'HUF', 'BGN', 'NMR', 'RLY', 'ZAR', 'PHP', 'XAF', 'XLM', 'ETH', 'BHD', 'KRW', 'IOTX', 'COMP', 'HTG', 'SDG', 'BZD', 'STD', 'IMP', 'KGS', ' ETH2', 'CNH', 'DOP', 'OMR', 'COP', 'SVC', 'RWF', 'CTX', 'GGP', 'ANKR', 'BOND', 'CUC', 'GNF', 'KHR', 'MKD', 'LTC', 'NEAR', 'NU', 'CUP', 'IRR', 'SZL', 'UGX', 'TMT', 'MUR', 'TWD', 'XDR', 'ACH', 'GTQ', 'BDT', 'NIO', 'AMP', 'LAK', 'SCR', 'MAD', 'BIF', 'SOS', 'XCD', 'STN', 'JMD', 'HOPR', 'CHZ', 'CLV', 'SAND', 'RON', 'ALGO', 'BTC', 'ERN', 'MNT', 'KZT', 'BND', 'MWK', 'GIP', 'BWP', 'MMK', 'THB', 'XPD', 'USDT', 'JPY', 'SSP', 'BRL', 'XPT', 'ZWL', 'RUB', 'SLL', 'ETH2', 'PEN', 'FKP', 'SEK', 'TOP', 'GMD', 'NPR', 'GAL', 'GYD', 'SHIB', 'BAL', 'ATOM', 'XBT', 'ALEPH', 'CVE', 'SAR', 'QNT', 'NAD', 'PKR', 'XOF', 'VES', 'PLN', 'GBP', 'GHS', 'RNDR', 'SYP', 'TJS', 'XCN', 'CLF', 'INR', 'PAB', 'LYD', 'RLC', 'NOK', 'KPW', 'XPF', 'SBD', 'LUNA', 'IDR', 'MYR', 'CAD', 'ACS', 'DKK', 'TND', 'AUCTION', 'LRD', 'VUV', 'HKD', 'MZN', 'DJF', 'TZS', 'BSD', 'VEF', 'IQD', 'TRY', 'KAVA', 'UZS', 'BTN', 'USD', 'BYN', 'MOP', 'BMD', 'KYD', 'ADA', 'CRC', 'ZMW', 'MGA', 'SKL', 'YER', 'CHF', 'SRD', 'CNY', 'HNL', 'GRT', 'KWD', 'ISK', 'CLP', 'UNI', 'UAH', 'CTSI', 'FJD', 'HRK', 'LKR', 'SGD', 'PGK', 'USDC', 'MXN', 'WST', 'MDL', 'LRC', 'TTD'}
2023-07-20 20:31:02,662/dali/DEBUG: Checking if EUR is in {'LBP', 'ETB', 'PYG', 'JEP', 'RSD', 'BOB', 'FORTH', 'XAG', 'QAR', 'DZD', 'SHP', 'FET', 'NZD', 'ILS', 'EGP', 'JOD', 'CDF', 'KMF', 'MRU', 'UMA', 'GEL', 'MRO', 'MVR', 'MATIC', 'NGN', 'CZK', 'KES', 'VND', 'XAU', 'UYU', 'EUR', 'LSL', 'HUF', 'BGN', 'NMR', 'RLY', 'ZAR', 'PHP', 'XAF', 'XLM', 'ETH', 'BHD', 'KRW', 'IOTX', 'COMP', 'HTG', 'SDG', 'BZD', 'STD', 'IMP', 'KGS', ' ETH2', 'CNH', 'DOP', 'OMR', 'COP', 'SVC', 'RWF', 'CTX', 'GGP', 'ANKR', 'BOND', 'CUC', 'GNF', 'KHR', 'MKD', 'LTC', 'NEAR', 'NU', 'CUP', 'IRR', 'SZL', 'UGX', 'TMT', 'MUR', 'TWD', 'XDR', 'ACH', 'GTQ', 'BDT', 'NIO', 'AMP', 'LAK', 'SCR', 'MAD', 'BIF', 'SOS', 'XCD', 'STN', 'JMD', 'HOPR', 'CHZ', 'CLV', 'SAND', 'RON', 'ALGO', 'BTC', 'ERN', 'MNT', 'KZT', 'BND', 'MWK', 'GIP', 'BWP', 'MMK', 'THB', 'XPD', 'USDT', 'JPY', 'SSP', 'BRL', 'XPT', 'ZWL', 'RUB', 'SLL', 'ETH2', 'PEN', 'FKP', 'SEK', 'TOP', 'GMD', 'NPR', 'GAL', 'GYD', 'SHIB', 'BAL', 'ATOM', 'XBT', 'ALEPH', 'CVE', 'SAR', 'QNT', 'NAD', 'PKR', 'XOF', 'VES', 'PLN', 'GBP', 'GHS', 'RNDR', 'SYP', 'TJS', 'XCN', 'CLF', 'INR', 'PAB', 'LYD', 'RLC', 'NOK', 'KPW', 'XPF', 'SBD', 'LUNA', 'IDR', 'MYR', 'CAD', 'ACS', 'DKK', 'TND', 'AUCTION', 'LRD', 'VUV', 'HKD', 'MZN', 'DJF', 'TZS', 'BSD', 'VEF', 'IQD', 'TRY', 'KAVA', 'UZS', 'BTN', 'USD', 'BYN', 'MOP', 'BMD', 'KYD', 'ADA', 'CRC', 'ZMW', 'MGA', 'SKL', 'YER', 'CHF', 'SRD', 'CNY', 'HNL', 'GRT', 'KWD', 'ISK', 'CLP', 'UNI', 'UAH', 'CTSI', 'FJD', 'HRK', 'LKR', 'SGD', 'PGK', 'USDC', 'MXN', 'WST', 'MDL', 'LRC', 'TTD'}
2023-07-20 20:31:02,946/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1m
2023-07-20 20:31:03,228/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 5m
2023-07-20 20:31:03,548/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 15m
2023-07-20 20:31:03,848/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1h
2023-07-20 20:31:04,157/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 4h
2023-07-20 20:31:04,453/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1d
2023-07-20 20:31:04,734/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1w
2023-07-20 20:31:04,734/CCXT-converter/nearest/DEBUG: No pricing data found for hop. This could be caused by airdropped
                    coins that do not have a market yet. Market - ETH2USDT, Timestamp - 2023-07-04 03:45:47+00:00, Exchange - Gate
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Retrieving bar for USDTUSD at 1688442347
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Retrieving cached bar for USDT, USD at 1688442347
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490807100 or 1688442347 > 1680307140
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490807100 or 1688442347 > 1680306900
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490806800 or 1688442347 > 1680306300
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490806800 or 1688442347 > 1680303600
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490788800 or 1688442347 > 1680264000
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490745600 or 1688442347 > 1680220800
2023-07-20 20:31:04,734/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1688442347 < 1490745600 or 1688442347 > 1680220800
2023-07-20 20:31:04,735/CCXT-converter/nearest/DEBUG: Delaying for 2.6281624794006344 seconds
2023-07-20 20:31:07,597/CCXT-converter/nearest/DEBUG: Got historical_data: [[1689834720000, 1.00009, 1.00009, 1.00007, 1.00007, 19159.58457406]] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1m
2023-07-20 20:31:07,597/CCXT-converter/nearest/DEBUG: For USDT/USD requested candle for 2023-07-04 03:45:47+00:00 (ms 1688442347000), but got 2023-07-20 06:32:00+00:00. Continuing with larger timeframe.
2023-07-20 20:31:07,597/CCXT-converter/nearest/DEBUG: Delaying for 4.866295194625854 seconds
2023-07-20 20:31:12,740/CCXT-converter/nearest/DEBUG: Got historical_data: [[1689662100000, 0.99998, 1.0, 0.99998, 0.99999, 235544.21711193]] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 5m
2023-07-20 20:31:12,741/CCXT-converter/nearest/DEBUG: For USDT/USD requested candle for 2023-07-04 03:45:47+00:00 (ms 1688442347000), but got 2023-07-18 06:35:00+00:00. Continuing with larger timeframe.
2023-07-20 20:31:12,741/CCXT-converter/nearest/DEBUG: Delaying for 4.822717761993408 seconds
2023-07-20 20:31:17,772/CCXT-converter/nearest/DEBUG: Got historical_data: [[1689230700000, 0.99988, 0.99989, 0.9998, 0.99986, 1807403.91159701]] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 15m
2023-07-20 20:31:17,772/CCXT-converter/nearest/DEBUG: For USDT/USD requested candle for 2023-07-04 03:45:47+00:00 (ms 1688442347000), but got 2023-07-13 06:45:00+00:00. Continuing with larger timeframe.
2023-07-20 20:31:17,773/CCXT-converter/nearest/DEBUG: Delaying for 4.891070461273193 seconds
2023-07-20 20:31:22,877/CCXT-converter/nearest/DEBUG: Got historical_data: [[1688443200000, 1.00004, 1.00016, 0.99997, 1.00004, 3206661.22514622]] with ms_timestamp - 1688442347000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1h
2023-07-20 20:31:23,064/dali/DEBUG: Fetched nearest conversion rate 0.91924876848 for 2023-07-04 03:45:47+00:00/ETH2->EUR from plugin CCXT-converter: HistoricalBar(duration=datetime.timedelta(days=1), timestamp=datetime.datetime(2023, 7, 4, 3, 45, 47, tzinfo=datetime.timezone.utc), open=Decimal('0.91924876848'), high=Decimal('0.91935907392'), low=Decimal('0.91918442364'), close=Decimal('0.91924876848'), volume=Decimal('3206661.22514622'))

here conf file.

[dali.plugin.pair_converter.ccxt]
historical_price_type = nearest
default_exchange = Gate
google_api_key = xxx
#untradeable_assets =
aliases = UNIVERSAL, ETH2, ETH, 1

maybe im doing something wrong :S

macanudo527 commented 1 year ago

@Larision Thanks for the log, you found a bug. I'll try to fix it this weekend and make a test.

By the way, the ETH2/ETH alias is already added, so you don't need to add it again. I'll update the docs with a list of the default aliases.

macanudo527 commented 1 year ago

In the future, I'd like there to be one big graph instead of tracking individual graphs for each exchange. I'm not sure if this is feasible for multiple reasons, but that is the current plan. It would make things a lot simpler.

Anyway, in order to implement 'teleportation' which is what I'm calling the process of stitching the graphs together to make one big graph, the mapped graph needs to know about exchanges. I was thinking that I would just extend the MappedGraph to allow for this functionality, but it seems like a subclass might be a better fit.

What I was trying to do with this PR is get out a fix without having to make another enormous PR since I have less and less time to work on this project. So, the MappedGraph ended up being a little halfway through the transition into a subclass to implement the 'super graph' made up of multiple graphs from multiple exchanges.

Ideally, what I would like to do is create a subclass that abstracts out the market tracking and graph tracking from ccxt. I just not sure if I have enough time to do it, and wanted something out there that works. Then we can address abstracting it out in a new PR to avoid code stink.

macanudo527 commented 1 year ago

@Larision I've been trying to recreate this bug, and built tests for it, but everything comes out okay. This line is the problem:

2023-07-20 20:31:02,662/CCXT-converter/nearest/DEBUG: Found path - ['ETH2', 'USDT', 'USD', 'EUR']

It should read ['ETH2', 'ETH', 'USDT', 'USD', 'EUR']

Can you search the log for 'ETH2', 'ETH' and see if it gets properly routed anywhere?

Also, can you try without declaring the aliases?

Larision commented 1 year ago

@macanudo527

Can you search the log for 'ETH2', 'ETH' and see if it gets properly routed anywhere?

No 'ETH2', 'ETH' in log. all coverts ['ETH2', 'USDT', 'USD', 'EUR']

Same results with or without aliases in .ini file.

Complete ini file is:

[dali.plugin.input.rest.coinbase fran]
account_holder = fran
api_key = xxx
api_secret = xxx
native_fiat = EUR

[dali.plugin.pair_converter.ccxt]
historical_price_type = nearest
default_exchange = Gate
google_api_key = xxx
;untradeable_assets =
aliases = UNIVERSAL, ETH2, ETH, 1

:(

macanudo527 commented 1 year ago

@Larision And you've tried with a different default exchange? I've spoofed my data to get ETH2 and the pricing works fine. I can't see anything in the code that would create a ETH2->USDT link.

macanudo527 commented 1 year ago

@Larision I've added temporary debugging logs to the code. Could you run it and let me know where you see "Found ETH2" in the logs? It should give you where and what pair like this:

Found ETH2 during cloning: ETH2 -> ETH
Larision commented 1 year ago

Tried kraken but giving me probkems with zip file data. I will try binance

macanudo527 commented 1 year ago

Tried kraken but giving me probkems with zip file data. I will try binance

Are you always getting the zipfile.BadZipFile: File is not a zip file error? Even on other branches?

You should only get zipfile.BadZipFile: File is not a zip file occasionally when the file is corrupted or the download gets interrupted.

Larision commented 1 year ago

yes only getting that error, im retrying until not showing error. What cache i should delete so dali not use cached conversions of assets? just dali_cache folder?

btw, im still testing eth2 issue, but im lack of time. i hope later can post some logs.

Larision commented 1 year ago

Ok, ive some logs to you @macanudo527.

-Gateio as default.

2023-08-03 16:41:54,628/dali/INFO: Country: es
2023-08-03 16:41:54,630/dali/DEBUG: InputPlugin object: '<dali.plugin.input.rest.coinbase.InputPlugin object at 0x7fdc07b5b9d0>'
2023-08-03 16:41:54,630/dali/INFO: Initialized input plugin 'dali.plugin.input.rest.coinbase fran'
2023-08-03 16:41:54,732/CCXT-converter/nearest/DEBUG: Default exchange assigned as Gate. _DEFAULT_EXCHANGE is Kraken
2023-08-03 16:41:54,732/dali/INFO: Initialized pair converter plugin 'dali.plugin.pair_converter.ccxt'
2023-08-03 16:41:54,732/dali/DEBUG: PairConverterPlugin object: '<dali.plugin.pair_converter.ccxt.PairConverterPlugin object at 0x7fdc07b6bdd0>'
2023-08-03 16:41:54,735/dali/INFO: Reading crypto data using plugin 'dali.plugin.input.rest.coinbase fran'

Found ETH2 during cloning found.

2023-08-03 16:43:39,379/dali/DEBUG: Found ETH2 in add_neighbor: ETH2 -> USDT
2023-08-03 16:43:39,379/dali/DEBUG: Found ETH2 during cloning: ETH2 -> USDT
2023-08-03 16:43:39,409/CCXT-converter/nearest/DEBUG: Added graph snapshot AVLTree for Gate for timestamp: 2021-02-24 00:00:00+00:00

lot of lines as this with 2 or 5 days date difference.

Now a really interesting in log. ADA conversion failing same as ETH2 :/

2023-08-03 16:43:49,692/CCXT-converter/nearest/DEBUG: Found path - ['ADA', 'USDT', 'USD', 'EUR']
2023-08-03 16:43:49,692/dali/DEBUG: Checking if ADA is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:43:49,692/dali/DEBUG: Checking if USDT is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:43:49,692/dali/DEBUG: Checking if USD is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:43:49,692/dali/DEBUG: Checking if EUR is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:43:49,976/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1m
2023-08-03 16:43:50,322/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 5m
2023-08-03 16:43:50,596/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 15m
2023-08-03 16:43:50,900/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1h
2023-08-03 16:43:51,170/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 4h
2023-08-03 16:43:51,443/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1d
2023-08-03 16:43:51,720/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1w
2023-08-03 16:43:51,720/CCXT-converter/nearest/DEBUG: No pricing data found for hop. This could be caused by airdropped
                    coins that do not have a market yet. Market - ADAUSDT, Timestamp - 2023-08-03 04:05:25+00:00, Exchange - Gate
2023-08-03 16:43:51,720/Kraken.com_CSVOHLCVT/DEBUG: Retrieving bar for USDTUSD at 1691035525
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Retrieving cached bar for USDT, USD at 1691035525
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490807100 or 1691035525 > 1680307140
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490807100 or 1691035525 > 1680306900
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490806800 or 1691035525 > 1680306300
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490806800 or 1691035525 > 1680303600
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490788800 or 1691035525 > 1680264000
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490745600 or 1691035525 > 1680220800
2023-08-03 16:43:51,721/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1691035525 < 1490745600 or 1691035525 > 1680220800
2023-08-03 16:43:51,721/CCXT-converter/nearest/DEBUG: Delaying for 0 seconds
2023-08-03 16:43:53,963/CCXT-converter/nearest/DEBUG: Got historical_data: [[1691035560000, 0.9993, 0.99931, 0.9993, 0.99931, 2465.66953007]] with ms_timestamp - 1691035525000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1m
2023-08-03 16:43:54,152/dali/DEBUG: Fetched nearest conversion rate 0.9149700723 for 2023-08-03 04:05:25+00:00/ADA->EUR from plugin CCXT-converter: HistoricalBar(duration=datetime.timedelta(days=1), timestamp=datetime.datetime(2023, 8, 3, 4, 5, 25, tzinfo=datetime.timezone.utc), open=Decimal('0.9149700723'), high=Decimal('0.91497922841'), low=Decimal('0.9149700723'), close=Decimal('0.91497922841'), volume=Decimal('2465.66953007'))
2023-08-03 16:43:54,153/dali/DEBUG: Self-contained transaction: InTransaction:
  plugin=Coinbase
  unique_id=xxxx-xxxx-xxxx-xxxx-xxxx
  raw_data={"id": "xxxx-xxxx-xxxx-xxxx-xxxx", "type": "staking_reward", "status": "completed", "amount": {"amount": "0.008598", "currency": "ADA"}, "native_amount": {"amount": "0.00", "currency": "EUR"}, "description": null, "created_at": "2023-08-03T04:05:25Z", "updated_at": "2023-08-03T04:05:25Z", "resource": "transaction", "resource_path": "/v2/accounts/xxxx-xxxx-xxxx-xxxx-xxxx/transactions/xxxx-xxxx-xxxx-xxxx-xxxx", "instant_exchange": false, "from": {"id": "xxxx-xxxx-xxxx-xxxx-xxxx", "resource": "user", "resource_path": "/v2/users/xxxxxxxxx-xxxx-xxxx-xxxx-xxxx", "currency": "ADA"}, "details": {"title": "Recompensa de Cardano", "subtitle": "De Coinbase", "header": "Se han recibido 0,008598 ADA (0,00 \u20ac)", "health": "positive"}, "hide_native_amount": false}
  timestamp=2023-08-03 04:05:25+0000
  asset=ADA
  exchange=Coinbase
  holder=fran
  transaction_type=Staking
  spot_price=0.9149700723
  crypto_in=0.008598
  fiat_fee=0
  fiat_in_no_fee=None
  fiat_in_with_fee=None
  notes=nearest spot_price read from CCXT-converter plugin; Recompensa de Cardano

Same problem in ETH2 ones:

2023-08-03 16:54:34,758/CCXT-converter/nearest/DEBUG: Converting ETH2 to EUR
2023-08-03 16:54:34,758/CCXT-converter/nearest/DEBUG: Using default exchange Gate type for Coinbase.
2023-08-03 16:54:34,764/CCXT-converter/nearest/DEBUG: Found path - ['ETH2', 'USDT', 'USD', 'EUR']
2023-08-03 16:54:34,764/dali/DEBUG: Checking if ETH2 is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:54:34,765/dali/DEBUG: Checking if USDT is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:54:34,765/dali/DEBUG: Checking if USD is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:54:34,765/dali/DEBUG: Checking if EUR is in {'XPD', 'AUCTION', 'TMT', 'UZS', 'IOTX', 'GGP', 'IQD', 'TOP', 'CLP', 'ILS', 'PYG', 'BTN', 'ACH', 'ETH', 'USDT', 'RLY', 'SGD', 'GHS', 'NEAR', 'ZMW', 'GRT', 'CNY', 'ATOM', 'MUR', 'BYN', 'SDG', 'XAF', 'ADA', 'QAR', 'CLV', 'FET', 'XCD', 'ZWL', 'ISK', 'UMA', 'MAD', 'COP', 'IMP', 'BSD', 'NOK', 'BMD', 'JPY', 'NIO', 'SHIB', 'GMD', 'EUR', 'PEN', 'HUF', 'XLM', 'MZN', 'RON', 'SLL', 'THB', 'KZT', 'SYP', 'ACS', 'COMP', 'KES', 'CLF', 'MWK', 'CAD', 'CTSI', 'LKR', 'LRC', 'GTQ', 'MVR', 'PHP', 'MRO', 'IRR', 'JMD', 'SZL', 'LBP', 'ETH2', 'KPW', 'UGX', 'PKR', 'DZD', 'XCN', 'GYD', 'ETB', 'SCR', 'IDR', 'TTD', 'MOP', 'GBP', 'SVC', 'XOF', 'FKP', 'BTC', 'MRU', 'NPR', 'TZS', 'XDR', 'QNT', 'CVE', 'LRD', 'USDC', 'MATIC', 'MDL', 'CUP', 'TJS', 'CNH', 'TWD', 'YER', 'RLC', 'KMF', 'VES', 'KAVA', 'MGA', 'AMP', 'BDT', 'LSL', 'BRL', 'PGK', 'NAD', 'UYU', 'LTC', 'SAR', 'HTG', 'MKD', 'UNI', 'KHR', 'SKL', 'BIF', 'USD', 'GEL', 'XAG', 'NZD', 'DOP', 'FJD', 'XPT', 'ALEPH', 'PAB', 'NMR', 'BGN', 'HKD', 'KRW', 'EGP', 'WST', 'INR', 'HOPR', 'RWF', 'BOB', 'XAU', 'ANKR', 'LUNA', 'KWD', 'CUC', 'CHF', 'NGN', 'ERN', 'BOND', 'LYD', 'VUV', 'CDF', 'RNDR', 'CZK', 'JOD', 'STD', 'XPF', 'SOS', 'TRY', 'KGS', 'SRD', 'SEK', 'GNF', 'PLN', 'STN', 'DKK', 'SBD', 'MYR', 'SSP', 'FORTH', 'GAL', 'GIP', 'MNT', 'LAK', 'ALGO', 'RSD', 'CHZ', 'HNL', 'HRK', 'UAH', 'MXN', 'SAND', 'CRC', 'JEP', 'BAL', 'BZD', 'ZAR', 'TND', 'XBT', 'BWP', 'BHD', 'KYD', 'MMK', 'RUB', 'VEF', 'SHP', 'VND', 'BND', 'OMR', 'DJF', ' ETH2'}
2023-08-03 16:54:35,054/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1m
2023-08-03 16:54:35,342/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 5m
2023-08-03 16:54:35,618/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 15m
2023-08-03 16:54:35,925/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1h
2023-08-03 16:54:36,197/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 4h
2023-08-03 16:54:36,474/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1d
2023-08-03 16:54:36,746/CCXT-converter/nearest/DEBUG: Got historical_data: [] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.gateio.gateio'> with timeframe - 1w
2023-08-03 16:54:36,746/CCXT-converter/nearest/DEBUG: No pricing data found for hop. This could be caused by airdropped
                    coins that do not have a market yet. Market - ETH2USDT, Timestamp - 2023-07-31 15:28:18+00:00, Exchange - Gate
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Retrieving bar for USDTUSD at 1690817298
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Retrieving cached bar for USDT, USD at 1690817298
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490807100 or 1690817298 > 1680307140
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490807100 or 1690817298 > 1680306900
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490806800 or 1690817298 > 1680306300
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490806800 or 1690817298 > 1680303600
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490788800 or 1690817298 > 1680264000
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490745600 or 1690817298 > 1680220800
2023-08-03 16:54:36,746/Kraken.com_CSVOHLCVT/DEBUG: Out of range - 1690817298 < 1490745600 or 1690817298 > 1680220800
2023-08-03 16:54:36,746/CCXT-converter/nearest/DEBUG: Delaying for 2.7075508594512936 seconds
2023-08-03 16:54:39,684/CCXT-converter/nearest/DEBUG: Got historical_data: [[1691031300000, 0.99937, 0.99937, 0.99936, 0.99936, 1221.02821321]] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1m
2023-08-03 16:54:39,684/CCXT-converter/nearest/DEBUG: For USDT/USD requested candle for 2023-07-31 15:28:18+00:00 (ms 1690817298000), but got 2023-08-03 02:55:00+00:00. Continuing with larger timeframe.
2023-08-03 16:54:39,684/CCXT-converter/nearest/DEBUG: Delaying for 4.869851684570312 seconds
2023-08-03 16:54:44,765/CCXT-converter/nearest/DEBUG: Got historical_data: [[1690858500000, 0.99978, 0.99978, 0.99976, 0.99976, 8735.92920754]] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 5m
2023-08-03 16:54:44,765/CCXT-converter/nearest/DEBUG: For USDT/USD requested candle for 2023-07-31 15:28:18+00:00 (ms 1690817298000), but got 2023-08-01 02:55:00+00:00. Continuing with larger timeframe.
2023-08-03 16:54:44,765/CCXT-converter/nearest/DEBUG: Delaying for 4.889212703704834 seconds
2023-08-03 16:54:49,874/CCXT-converter/nearest/DEBUG: Got historical_data: [[1690817400000, 0.99972, 0.9998, 0.99971, 0.99979, 736559.67497136]] with ms_timestamp - 1690817298000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 15m
2023-08-03 16:54:50,065/dali/DEBUG: Fetched nearest conversion rate 0.90916936128 for 2023-07-31 15:28:18+00:00/ETH2->EUR from plugin CCXT-converter: HistoricalBar(duration=datetime.timedelta(days=1), timestamp=datetime.datetime(2023, 7, 31, 15, 28, 18, tzinfo=datetime.timezone.utc), open=Decimal('0.90916936128'), high=Decimal('0.9092421152'), low=Decimal('0.90916026704'), close=Decimal('0.90923302096'), volume=Decimal('736559.67497136'))
2023-08-03 16:54:50,065/dali/DEBUG: Self-contained transaction: InTransaction:
  plugin=Coinbase
  unique_id=xxxxxxxxx-xxxx-xxxx-xxxx-xxxx
  raw_data={"id": "xxxxxxxxx-xxxx-xxxx-xxxx-xxxx", "type": "staking_reward", "status": "completed", "amount": {"amount": "0.00000292", "currency": "ETH2"}, "native_amount": {"amount": "0.00", "currency": "EUR"}, "description": null, "created_at": "2023-07-31T15:28:18Z", "updated_at": "2023-07-31T15:28:18Z", "resource": "transaction", "resource_path": "/v2/accounts/xxxxxxxxx-xxxx-xxxx-xxxx-xxxx/transactions/xxxxxxxxx-xxxx-xxxx-xxxx-xxxx", "instant_exchange": false, "from": {"id": "xxxxxxxxx-xxxx-xxxx-xxxx-xxxx", "resource": "user", "resource_path": "/v2/users/xxxxxxxxx-xxxx-xxxx-xxxx-xxxx", "currency": "ETH2"}, "details": {"title": "Recompensa de Ethereum 2", "subtitle": "De Coinbase", "header": "Se han recibido 0,00000292 ETH2 (0,00 \u20ac)", "health": "positive"}, "hide_native_amount": false}
  timestamp=2023-07-31 15:28:18+0000
  asset=ETH2
  exchange=Coinbase
  holder=fran
  transaction_type=Staking
  spot_price=0.90916936128
  crypto_in=0.00000292
  fiat_fee=0
  fiat_in_no_fee=None
  fiat_in_with_fee=None
  notes=nearest spot_price read from CCXT-converter plugin; Recompensa de Ethereum 2

-Kraken and binance.com as default.

2023-08-02 22:28:48,685/CCXT-converter/nearest/DEBUG: Converting ETH2 to EUR
2023-08-02 22:28:48,685/CCXT-converter/nearest/DEBUG: Using default exchange Binance.com type for Coinbase.
2023-08-02 22:28:48,686/dali/ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/dali_main.py", line 184, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 138, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 110, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/abstract_pair_converter_plugin.py", line 163, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 324, in get_historic_bar_from_native_source
    raise RP2RuntimeError(f"The asset {from_asset} or {to_asset} is missing from graph")
rp2.rp2_error.RP2RuntimeError: The asset ETH2 or EUR is missing from graph
2023-08-02 22:28:48,687/dali/INFO: Log file: ./log/rp2_2023_08_02_22_26_16_505246.log
2023-08-02 22:28:48,687/dali/INFO: Generated output directory: salida/
2023-08-02 22:28:48,687/dali/INFO: Done
2023-08-03 16:28:42,647/CCXT-converter/nearest/DEBUG: Converting ETH2 to EUR
2023-08-03 16:28:42,647/CCXT-converter/nearest/DEBUG: Using default exchange Kraken type for Coinbase.
2023-08-03 16:28:42,647/dali/ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/dali_main.py", line 184, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 138, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 110, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/abstract_pair_converter_plugin.py", line 163, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 324, in get_historic_bar_from_native_source
    raise RP2RuntimeError(f"The asset {from_asset} or {to_asset} is missing from graph")
rp2.rp2_error.RP2RuntimeError: The asset ETH2 or EUR is missing from graph
2023-08-03 16:28:42,649/dali/INFO: Log file: ./log/rp2_2023_08_03_16_09_03_159322.log
2023-08-03 16:28:42,649/dali/INFO: Generated output directory: salida/
2023-08-03 16:28:42,649/dali/INFO: Done

Found ETH2 during cloning:

not found.

Huobi as default exchange:

2023-08-02 22:25:51,081/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - huobi {"ts":1691007950988,"status":"error","err-code":"invalid-parameter","err-msg":"invalid size,valid range: [1, 1000]"}
2023-08-02 22:25:51,415/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - huobi {"ts":1691007951320,"status":"error","err-code":"invalid-parameter","err-msg":"invalid symbol"}

lot of those errors and this one:

2023-08-02 22:25:52,217/CCXT-converter/nearest/DEBUG: Added graph snapshot AVLTree for Huobi for timestamp: 2021-02-25 00:00:00+00:00
2023-08-02 22:25:52,223/CCXT-converter/nearest/DEBUG: Found path - ['ADA', 'USDT', 'USD', 'EUR']
2023-08-02 22:25:52,223/dali/DEBUG: Checking if ADA is in {'BRL', 'STD', 'SVC', 'XAF', 'RSD', 'KHR', 'SAR', 'JPY', 'ZWL', 'GMD', 'MGA', 'GNF', 'TZS', 'LRD', 'MZN', 'TTD', 'XCD', 'LYD', 'SBD', 'KMF', 'XAG', 'XPF', 'HNL', 'SGD', 'KWD', 'BZD', 'LAK', 'LSL', 'MVR', 'MRO', 'THB', 'NGN', 'MXN', 'IQD', 'VES', 'GIP', 'DOP', 'ERN', 'BHD', 'XPD', 'SDG', 'TWD', 'DKK', 'JEP', 'IMP', 'PEN', 'PKR', 'GBP', 'HTG', 'PYG', 'UZS', 'LUNA', 'RON', 'IDR', 'UAH', 'PGK', 'CNY', 'ZAR', 'CHF', 'SLL', 'GHS', 'MRU', 'KPW', 'PHP', 'IRR', 'MWK', 'BTN', 'TRY', 'USD', 'HKD', 'KES', 'UGX', 'CUC', 'NZD', 'HRK', 'PAB', 'CLF', 'LBP', 'MMK', 'KZT', 'BSD', 'SRD', 'XBT', 'GEL', 'CLP', 'CUP', 'VND', 'CNH', 'KYD', 'EUR', 'STN', 'HUF', 'CAD', 'VEF', 'LKR', 'GTQ', 'MKD', 'SYP', 'SOS', 'NOK', 'ISK', 'DZD', 'TND', 'WST', 'BWP', 'XAU', 'CZK', 'YER', 'GYD', 'NIO', 'SCR', 'BMD', 'MYR', 'TOP', 'MDL', 'PLN', 'SZL', 'CVE', 'XDR', 'MAD', 'ILS', 'KGS', 'DJF', 'FKP', 'MOP', 'NPR', 'SEK', 'SHP', 'TMT', 'FJD', 'TJS', 'CDF', 'RWF', 'ZMW', 'BOB', 'CRC', 'JMD', 'GGP', 'ETB', 'XOF', 'XPT', 'MNT', 'OMR', 'UYU', 'NAD', 'BND', 'BIF', 'RUB', 'BYN', 'QAR', 'INR', 'MUR', 'USDT', 'KRW', 'COP', 'JOD', 'SSP', 'EGP', 'VUV'}
2023-08-02 22:25:52,223/dali/ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/dali_main.py", line 184, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 138, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/transaction_resolver.py", line 110, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/abstract_pair_converter_plugin.py", line 163, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali_macanudo/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 336, in get_historic_bar_from_native_source
    raise RP2RuntimeError(f"Internal Error: The asset {asset} is not optimized.")
rp2.rp2_error.RP2RuntimeError: Internal Error: The asset ADA is not optimized.
2023-08-02 22:25:52,224/dali/INFO: Log file: ./log/rp2_2023_08_02_22_23_35_904974.log
2023-08-02 22:25:52,224/dali/INFO: Generated output directory: salida/
2023-08-02 22:25:52,224/dali/INFO: Done

i hope this can help you.

macanudo527 commented 1 year ago

@Larision You never saw Found ETH2 while adding alternative markets in any log?

Larision commented 1 year ago

not in binance, kraken and huobi logs. this one in gateio log.

2023-08-02 21:00:50,212/CCXT-converter/nearest/DEBUG: Market: {'id': 'ETH2_USDT', 'symbol': 'ETH2/USDT', 'base': 'ETH2', 'quote': 'USDT', 'settle': None, 'baseId': 'ETH2', 'quoteId': 'USDT', 'settleId': None, 'type': 'spot', 'spot': True, 'margin': False, 'swap': False, 'future': False, 'option': False, 'active': True, 'contract': False, 'linear': None, 'inverse': None, 'taker': 0.002, 'maker': 0.002, 'contractSize': None, 'expiry': None, 'expiryDatetime': None, 'strike': None, 'optionType': None, 'precision': {'amount': 0.0001, 'price': 0.001}, 'limits': {'leverage': {'min': 1.0, 'max': 1}, 'amount': {'min': 0.0001, 'max': None}, 'price': {'min': None, 'max': None}, 'cost': {'min': 1.0, 'max': None}}, 'info': {'id': 'ETH2_USDT', 'base': 'ETH2', 'quote': 'USDT', 'fee': '0.2', 'min_quote_amount': '1', 'amount_precision': '4', 'precision': '3', 'trade_status': 'tradable', 'sell_start': '0', 'buy_start': '1606406400'}}
2023-08-02 21:00:50,212/dali/DEBUG: Found ETH2 in add_neighbor: ETH2 -> USDT
2023-08-02 21:00:50,212/CCXT-converter/nearest/DEBUG: Found ETH2 while adding alternative markets: ETH2 -> USDT

if you want full logs tell me.

macanudo527 commented 1 year ago

@Larision I have some good? news I guess. I can get these errors locally on my computer, now. I can replicate the same situation you have. Hopefully, I can work on it faster on my own by myself. I should be able to find the bug faster. Thanks for all your help. We will fix it soon!

macanudo527 commented 1 year ago

@Larision Can you give it a try again? This seems to be working for me.

@eprbell I added a feature to the RP2 ODS input plugin "force-repricing". This essentially allows me to spoof transactions from any exchange I like for testing. Let me know if I should put it in a separate PR.

eprbell commented 1 year ago

@Larision Can you give it a try again? This seems to be working for me.

@eprbell I added a feature to the RP2 ODS input plugin "force-repricing". This essentially allows me to spoof transactions from any exchange I like for testing. Let me know if I should put it in a separate PR.

No worries, it's OK to leave it here. Let me know when you're ready for another round of review.

Larision commented 1 year ago 
plugin=Coinbase
  unique_id=bd1a2d36-f7e2-58be-99c2-c4f8f38b1e22
  raw_data={"id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx", "type": "inflation_reward", "status": "completed", "amount": {"amount": "0.004534", "currency": "ALGO"}, "native_amount": {"amount": "0.00", "currency": "EUR"}, "description": null, "created_at": "2021-07-13T10:43:57Z", "updated_at": "2021-11-10T20:45:03Z", "resource": "transaction", "resource_path": "/v2/accounts/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/transactions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx", "instant_exchange": false, "from": {"id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx", "resource": "user", "resource_path": "/v2/users/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx", "currency": "ALGO"}, "details": {"title": "Recompensa de Algorand", "subtitle": "De Coinbase", "header": "Se han recibido 0,004534 ALGO (0,00 \u20ac)", "health": "positive"}, "hide_native_amount": false}
  timestamp=2021-07-13 10:43:57+0000
  asset=ALGO
  exchange=Coinbase
  holder=fran
  transaction_type=Income
  spot_price=0.72569
  crypto_in=0.004534
  fiat_fee=0
  fiat_in_no_fee=None
  fiat_in_with_fee=None
  notes=nearest spot_price read from CCXT-converter plugin; Recompensa de Algorand
2023-08-08 22:43:19,377/CCXT-converter/nearest/DEBUG: Converting ALGO to EUR
2023-08-08 22:43:19,377/CCXT-converter/nearest/DEBUG: Found market - ALGOEUR on single exchange, skipping routing.
2023-08-08 22:43:20,396/CCXT-converter/nearest/DEBUG: Got historical_data: [[1691484240000, 0.1026, 0.1026, 0.1026, 0.1026, 0.0]] with ms_timestamp - 1626095530000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1m
2023-08-08 22:43:20,396/CCXT-converter/nearest/DEBUG: For ALGO/EUR requested candle for 2021-07-12 13:12:10+00:00 (ms 1626095530000), but got 2023-08-08 08:44:00+00:00. Continuing with larger timeframe.
2023-08-08 22:43:21,410/CCXT-converter/nearest/DEBUG: Got historical_data: [[1691311500000, 0.09797, 0.09797, 0.09797, 0.09797, 84.28794279]] with ms_timestamp - 1626095530000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 5m
2023-08-08 22:43:21,410/CCXT-converter/nearest/DEBUG: For ALGO/EUR requested candle for 2021-07-12 13:12:10+00:00 (ms 1626095530000), but got 2023-08-06 08:45:00+00:00. Continuing with larger timeframe.
2023-08-08 22:43:22,361/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:23,401/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:24,371/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:25,415/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:26,362/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:27,534/CCXT-converter/nearest/DEBUG: Got historical_data: [[1688936400000, 0.10154, 0.10154, 0.101, 0.101, 14923.42622314]] with ms_timestamp - 1626095530000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1h
2023-08-08 22:43:27,534/CCXT-converter/nearest/DEBUG: For ALGO/EUR requested candle for 2021-07-12 13:12:10+00:00 (ms 1626095530000), but got 2023-07-09 21:00:00+00:00. Continuing with larger timeframe.
2023-08-08 22:43:28,664/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:29,653/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:30,369/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:31,361/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:32,431/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:33,381/CCXT-converter/nearest/DEBUG: Got historical_data: [[1629331200000, 0.7901, 0.91567, 0.78252, 0.89394, 1784350.3557375]] with ms_timestamp - 1626095530000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1d
2023-08-08 22:43:33,382/CCXT-converter/nearest/DEBUG: For ALGO/EUR requested candle for 2021-07-12 13:12:10+00:00 (ms 1626095530000), but got 2021-08-19 00:00:00+00:00. Continuing with larger timeframe.
2023-08-08 22:43:34,371/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:35,361/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:36,377/CCXT-converter/nearest/DEBUG: Exception from server, most likely too many requests. Making another attempt after 0.1 second delay. Exception - kraken {"error":["EGeneral:Too many requests"]}
2023-08-08 22:43:36,479/dali/ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/kali/dali-rp2/src/dali/dali_main.py", line 184, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/transaction_resolver.py", line 146, in _update_spot_price_from_web
    raise RP2RuntimeError(
rp2.rp2_error.RP2RuntimeError: Spot price for xxxxxxxx-xxxx-xxxx-xxxx-xxxx:2021-07-12 13:12:10+00:00:ALGO->EUR not found on any pair converter plugin
2023-08-08 22:43:36,486/dali/INFO: Log file: ./log/rp2_2023_08_08_22_06_31_521639.log
2023-08-08 22:43:36,486/dali/INFO: Generated output directory: salida/
2023-08-08 22:43:36,486/dali/INFO: Done

ETH2 converting is ok now. too many requests for kraken all along log. wanna i try with other exchanges? this one is kraken one.

Larision commented 1 year ago 
INFO: Building manifest to optimize price calculation with the pair converters.
INFO: Resolving transactions
  0% |                                                                                                                                                                                                                                                       | Elapsed Time: 0:00:00 ETA:  --:--:--
ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/kali/dali-rp2/src/dali/dali_main.py", line 184, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/transaction_resolver.py", line 138, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/transaction_resolver.py", line 110, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/abstract_pair_converter_plugin.py", line 163, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kali/dali-rp2/src/dali/plugin/pair_converter/ccxt.py", line 365, in get_historic_bar_from_native_source
    duration=max(result.duration, hop_bar.duration),  # type: ignore
                                  ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'duration'
INFO: Log file: ./log/rp2_2023_08_08_23_39_22_174494.log
INFO: Generated output directory: salida/
INFO: Done

with gate exchange getting this error.

macanudo527 commented 1 year ago

ETH2 converting is ok now. too many requests for kraken all along log. wanna i try with other exchanges? this one is kraken one.

This is an error with ALGO, which should be unrelated. Are you getting this error on the main branch, too?

Larision commented 1 year ago 
2023-08-09 17:17:30,501/CCXT-converter/nearest/DEBUG: Delaying for 4.8898530960083 seconds
2023-08-09 17:17:35,603/CCXT-converter/nearest/DEBUG: Got historical_data: [[1691551080000, 0.270189, 0.270189, 0.270189, 0.270189, 0.0]] with ms_timestamp - 1686268361000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1m
2023-08-09 17:17:35,603/CCXT-converter/nearest/DEBUG: For ADA/EUR requested candle for 2023-06-08 23:52:41+00:00 (ms 1686268361000), but got 2023-08-09 03:18:00+00:00. Continuing with larger timeframe.
2023-08-09 17:17:35,603/CCXT-converter/nearest/DEBUG: Delaying for 4.888081169128418 seconds
2023-08-09 17:17:40,784/CCXT-converter/nearest/DEBUG: Got historical_data: [[1691378400000, 0.267613, 0.267613, 0.267573, 0.267573, 4251.81563164]] with ms_timestamp - 1686268361000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 5m
2023-08-09 17:17:40,784/CCXT-converter/nearest/DEBUG: For ADA/EUR requested candle for 2023-06-08 23:52:41+00:00 (ms 1686268361000), but got 2023-08-07 03:20:00+00:00. Continuing with larger timeframe.
2023-08-09 17:17:40,784/CCXT-converter/nearest/DEBUG: Delaying for 4.807214593887329 seconds
2023-08-09 17:17:45,843/CCXT-converter/nearest/DEBUG: Got historical_data: [[1690947000000, 0.2802, 0.2802, 0.279966, 0.279966, 5246.95325608]] with ms_timestamp - 1686268361000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 15m
2023-08-09 17:17:45,843/CCXT-converter/nearest/DEBUG: For ADA/EUR requested candle for 2023-06-08 23:52:41+00:00 (ms 1686268361000), but got 2023-08-02 03:30:00+00:00. Continuing with larger timeframe.
2023-08-09 17:17:45,843/CCXT-converter/nearest/DEBUG: Delaying for 4.8483868598937985 seconds
2023-08-09 17:17:51,201/CCXT-converter/nearest/DEBUG: Got historical_data: [[1689004800000, 0.262104, 0.262538, 0.26051, 0.261376, 35227.67238931]] with ms_timestamp - 1686268361000 with exchange <class 'ccxt.kraken.kraken'> with timeframe - 1h
2023-08-09 17:17:51,201/CCXT-converter/nearest/DEBUG: For ADA/EUR requested candle for 2023-06-08 23:52:41+00:00 (ms 1686268361000), but got 2023-07-10 16:00:00+00:00. Continuing with larger timeframe.
2023-08-09 17:17:51,201/CCXT-converter/nearest/DEBUG: Delaying for 4.591031646728515 seconds

seems main branch delaying requests instead retrying too quick isnt? thats why i didnt notice before..

Larision commented 1 year ago

This is an error with ALGO, which should be unrelated. Are you getting this error on the main branch, too?

You mean ALGO error or requests one?

macanudo527 commented 1 year ago

The delaying on the main branch is not a bug. Kraken only allows one request every 5 seconds. If you make too many requests, Kraken might ban your IP temporarily.

You might not have noticed it before because it only needs to request the last quarter of prices. If you delete the .dali_cache folder you should be able to get the latest prices.

macanudo527 commented 1 year ago

This LGTM. However I'm wondering if force_repricing should override the read_spot_price_from_web option.

In other words, what's the behavior we want to happen if the user sets force_repricing to true but doesn't use -s?

Yeah, if you don't set the -s flag, it doesn't request prices at all. Should we rename force_repricing to delete_pricing_info then we wouldn't have the unexpected behavior of reading spot prices from the web without explicitly declaring it.

eprbell commented 1 year ago

I think it's fine to call it force_repricing, however let's add to the documentation that the user is supposed to add the -s flag to see results. We could also add a warning in main if force_repricing is set but -s is not.

macanudo527 commented 1 year ago

@eprbell I added clarification in the latest commit.