ccxt / ccxt

A JavaScript / TypeScript / Python / C# / PHP cryptocurrency trading API with support for more than 100 bitcoin/altcoin exchanges
https://docs.ccxt.com
MIT License
32.81k stars 7.5k forks source link

"Ping-pong keep alive missing on time lead" to "throttle queue is over maxCapacity" in bybit ccxt pro on watchOHLC #23929

Open gaugau3000 opened 1 week ago

gaugau3000 commented 1 week ago

Operating System

Linux

Programming Languages

JavaScript

CCXT Version

4.4.10

Description

Workflow :

1/ Connection to 416 bybit linear watch candle on ccxt pro on 1m timeframe. 2/ Loose ping-pong on the 416 ws conn.

Error with : Connection to wss://stream.bybit.com/v5/public/linear timed out due to a ping-pong keepalive missing on time for THETA/USDT:USDT

3/ the retry lead to:

Error with : throttle queue is over maxCapacity (1000), see https://github.com/ccxt/ccxt/issues/11645#issuecomment-1195695526 for GOMINING/USDT:USDT

4/ the program lead to memory leak.

Here I am not sure why the queue is over capacity because the retry should not really lead to increase it (keep at 416) ?

Many TKS.

Code

async wsCandlesSubscribe(candlesManagersForPair: CandlesManager[], relatedPair: string): Promise<void> {
        while (true) {
            try {
                const snapshot1mCandles = await this.ccxtExchange.watchOHLCV(relatedPair, "1m") as OHLCV[];
                for (const candlesManagerForPairTf of candlesManagersForPair) {
                    this.processCandle1mSnapshot(snapshot1mCandles, candlesManagerForPairTf);
                }
            } catch (error) {
                this.handleError(error, relatedPair);
            }
        }
    }
carlosmiei commented 1 week ago

Hello @gaugau3000,

Thanks for reporting. Instead of calling watchOHLCV 416 times, can you try to use watchOHLCVForSymbols instead?

It should be used like ex.watchOHLCVForSymbols([[symbol, timeframe], [symbol2, params2], params)

carlosmiei commented 1 week ago

Nonetheless I will try to check the issue you're facing.

gaugau3000 commented 1 week ago

Hello @gaugau3000,

Thanks for reporting. Instead of calling watchOHLCV 416 times, can you try to use watchOHLCVForSymbols instead?

It should be used like ex.watchOHLCVForSymbols([[symbol, timeframe], [symbol2, params2], params)

Yes, will give a try, many TKS (I open one connection for each symbol vs on connection with n symbols with is poor choice design on my side maybe).

carlosmiei commented 1 week ago

@gaugau3000 The connection is always reused so you have only one connection regardless of the approach, however sending hundreds of messages might be a problem.