LBANK: Outdated history

[abmahmoodi]

Describe your environment

• Operating system: Ubuntu 22.04
• Python Version: 3.10.12
• CCXT version: ccxt==4.4.6
• Freqtrade Version: 2024.9

Your question

I have the following message in logs: WARNING - Outdated history for pair DOGE/USDT. Last tick is 707 minutes old

I have checked the data by the following api: https://api.lbank.info/v2/kline.do?size=500&symbol=doge_usdt&time=1725667200&type=hour4

The output in up to date.

My server clock in synced.

Please help to find problem.

[xmatthias]

In theory, freqtrade supports every exchange ccxt supports, which would also include this exchange.

Now you're the first to ask for that (at least as far as i remember), so you're probably the first to have a look at this. we have a rough guideline on things to check in the contributors section of the documentation. While it's probably not complete, it should give you a good idea if the exchange will work for spot trading.

If you encounter specific problems, we're happy to assist up to a certain point - but the expectation for new exchanges is that the user interested is doing the legwork / most of the testing work - which also assumes a somewhat familiarity with python / testing/debugging things in python.

If you're new to python / development - you're almost certainly better of with one of the supported exchanges.

---

For this specific exchange - you seem to get only closed candles (but not the currently opened candle). While it's visible on 4h, but more obvious on 1h candles, so the below sample uses 1h for simplicity.

import ccxt
from datetime import datetime, timezone
exchange = ccxt.lbank()
_ = exchange.load_markets()
print(ccxt.__version__)

# exchange.verbose = True

x = exchange.fetch_ohlcv('BTC/USDT', timeframe='1h', limit=500,)

print(f"now: {datetime.now(tz=timezone.utc)}")
print(f"first date: {x[0][0]}, {datetime.fromtimestamp(x[0][0] // 1000, tz=timezone.utc)}")
print(f"last date : {x[-1][0]} {datetime.fromtimestamp(x[-1][0] // 1000, tz=timezone.utc)}")
len(x)

now: 2024-10-16 04:48:36.125664+00:00
first date: 1727254800000, 2024-09-25 09:00:00+00:00
last date : 1729047600000 2024-10-16 03:00:00+00:00

Having the current data within the 04:00 candle - the 3:00 candle is the last closed candle - but by default, ccxt provides the unclosed candle, too. This seems to align with what the exchange provides (only close candles, not the currently opened candle) - but it deviates from how most other exchanges behave.

The above mentioned developer docs do mention how to check for this - as well as what needs to be changed to facilitate this (essentially it's "subclass the exchange, add it to __init__, override a few settings" - which one is up to experimentation i'd say).

Happy to get PR's for this exchange if you feel like it - otherwise best stick with supported / tested exchanges.

[abmahmoodi]

Can I use this exchange with Freqtrade? Or should the problems be solved first?

[xmatthias]

Outdated history will invalidate all signals your strategy creates - so you'll definitely need to work on the problems first.

This may also not be the last problem you encounter - as said, you're the first to ever attempt to use lbank (I'm pretty certain about that - as it's the first time i hear the name of this exchange).

[abmahmoodi]

I investigated this problem more closely. For example, in the 15-minute time frame, when the previous candle closes, this warning is not displayed for 5 minutes.

[xmatthias]

you're looking at the wrong thing, really. the warning is shown after a specified grace period - the PROBLEM is that the exchange does deviate from standard behavior and doesn't provide the non-closed candle. That's something freqtrade can handle - but needs per exchange configuration (see the above linked docs as to why).

[abmahmoodi]

Please get an address to Freqtrade source code to fix this problem for LBank exchange. I am Ruby and C# developer and I can work on this issue.

[xmatthias]

Look at the exchange directory within the freqtrade module (the information as to what is required you'll find in the link i provided above

[abmahmoodi]

I have checked the source code:

latest_date = dataframe["date"].max()
        latest = dataframe.loc[dataframe["date"] == latest_date].iloc[-1]
        # Explicitly convert to datetime object to ensure the below comparison does not fail
        latest_date = latest_date.to_pydatetime()

        # Check if dataframe is out of date
        timeframe_minutes = timeframe_to_minutes(timeframe)
        offset = self.config.get("exchange", {}).get("outdated_offset", 5)
        if latest_date < (dt_now() - timedelta(minutes=timeframe_minutes * 2 + offset)):
            logger.warning(
                "Outdated history for pair %s. Last tick is %s minutes old",
                pair,
                int((dt_now() - latest_date).total_seconds() // 60),
            )
            return None, None

and create a log of data as below:

time frame minutes: 15 OFFSET: 5 timedelta: 2024-10-16 10:29:58.840384+00:00 latest_date: 2024-10-16 10:30:00+00:00 latest DATA: date 2024-10-16 10:30:00+00:00

in this line of code latest is not less than (dt_now() - timedelta(minutes=timeframe_minutes 2 + offset)) ` if latest_date < (dt_now() - timedelta(minutes=timeframe_minutes 2 + offset)):`

I think we have a bug here.

[abmahmoodi]

I use 15 minutes time frame, and the library gives an error after 5 minutes, that is, the offset, but the next candle closes after 15 minutes. I think I should increase the offset value in the settings

[xmatthias]

There is neither a bug, nor anything wrong with the offset. It's the exchange data that's wrong (not aligned to how ALL other exchanges provide the data).

The new exchange documentation contains an explicit point about exactly this topic (complete / incomplete data) - but the conversation here makes me think that you probably never even clicked that link - or at least didn't read it's contents.

The steps have been outlined in this comment: https://github.com/freqtrade/freqtrade/issues/10803#issuecomment-2415728842 - combine that with the information from the linked developer docs - and it should get you starting (you say you have a development background - the language is not really relevant).

If you can't get anywhere with the instructions provided in that comment and with the docs - then please use one of the supported exchanges.