Unstable latency of Websocket watch_ticker

Crypto7816 commented 1 week ago

Operating System

linux ubuntu 22.04

Programming Languages

Python

CCXT Version

4.3.47

Description

There is unstable latency of watch_ticker. For instance, subscribing to BTC/USDT:USDT can result in a latency of up to 1351 ms. How can I minimize this latency? While testing on cloudzy, the ping latency to api.binance.com is below 2ms, so I am curious why this latency could extend to 1s.

You can produce this by running python test_lantency.py -s linear -e binance

symbol: BTC/USDT:USDT delay: 553
symbol: BTC/USDT:USDT delay: 58
symbol: BTC/USDT:USDT delay: 76
symbol: BTC/USDT:USDT delay: 109
symbol: BTC/USDT:USDT delay: 312
symbol: BTC/USDT:USDT delay: 261
symbol: BTC/USDT:USDT delay: 486
symbol: BTC/USDT:USDT delay: 329
symbol: BTC/USDT:USDT delay: 131
symbol: BTC/USDT:USDT delay: 465
symbol: BTC/USDT:USDT delay: 587
symbol: BTC/USDT:USDT delay: 186
symbol: BTC/USDT:USDT delay: 125
symbol: BTC/USDT:USDT delay: 116
symbol: BTC/USDT:USDT delay: 290
symbol: BTC/USDT:USDT delay: 155
symbol: BTC/USDT:USDT delay: 155
symbol: BTC/USDT:USDT delay: 505
symbol: BTC/USDT:USDT delay: 184
symbol: BTC/USDT:USDT delay: 459
symbol: BTC/USDT:USDT delay: 204
symbol: BTC/USDT:USDT delay: 680
symbol: BTC/USDT:USDT delay: 284
symbol: BTC/USDT:USDT delay: 90
symbol: BTC/USDT:USDT delay: 1351
symbol: BTC/USDT:USDT delay: 114
symbol: BTC/USDT:USDT delay: 111
symbol: BTC/USDT:USDT delay: 147
symbol: BTC/USDT:USDT delay: 533
symbol: BTC/USDT:USDT delay: 300
symbol: BTC/USDT:USDT delay: 105
symbol: BTC/USDT:USDT delay: 235
symbol: BTC/USDT:USDT delay: 454
symbol: BTC/USDT:USDT delay: 423
symbol: BTC/USDT:USDT delay: 146
symbol: BTC/USDT:USDT delay: 69
symbol: BTC/USDT:USDT delay: 75

Code

import ccxt.pro as ccxtpro
import ccxt
import asyncio
from nats.aio.client import Client as NATS
import argparse
from typing import Literal, List
import time

def arg_parse():
    # Define a list of valid exchange IDs
    valid_exchange_ids = ['binance', 'okx']

    # Define a list of valid symbol types
    valid_symbol_types = ['spot', 'linear']

    parser = argparse.ArgumentParser(description="publish market data")
    parser.add_argument("-s", "--symbol_type", default="", required=True, help="spot/linear/inverse/future")
    parser.add_argument("-e", "--exchange_id", default="", required=True, help="exchange id, see CCXT")

    args = parser.parse_args()

    # Check if the provided exchange_id is in the valid list
    if args.exchange_id not in valid_exchange_ids:
        parser.error(f"Invalid exchange ID '{args.exchange_id}'. Please use one of the following: {', '.join(valid_exchange_ids)}")

    # Check if the provided symbol_type is in the valid list
    if args.symbol_type not in valid_symbol_types:
        parser.error(f"Invalid symbol type '{args.symbol_type}'. Please use one of the following: {', '.join(valid_symbol_types)}")

    return args

def get_type(exchange: ccxt.Exchange, tp: Literal['spot', 'linear']):
    exchange.load_markets()
    spots = []
    linears = []
    for symbol, data in exchange.markets.items():
        if data['active'] and data['quote'] == 'USDT':
            if data['type'] == 'spot':
                spots.append(symbol)
            elif data['type'] == 'swap':
                linears.append(symbol)
    if tp == 'spot':
        return spots
    elif tp == 'linear':
        return linears

async def watch_tickers(exchange: ccxtpro.Exchange, symbols: List[str]):
    while True:
        tickers = await exchange.watch_tickers(symbols)
        for symbol, value in tickers.items():
            # print("%s.%s.ticker.%s"%(exchange.id, tp, symbol), value)
            if ":" in symbol:
                type_ = "linear"
            else:
                type_ = "spot" 

            local_time = int(round(time.time() * 1000))
            delay = local_time - value['timestamp'] 
            print(f'symbol: {symbol} delay: {delay}')

async def main():
    args = arg_parse()

    config = {
        'exchange_id': args.exchange_id,
    }
    exchange = getattr(ccxt, config['exchange_id'])(config)
    exchange_pro = getattr(ccxtpro, config['exchange_id'])(config)
    exchange_pro.enableRateLimit = False

    symbols = get_type(exchange, args.symbol_type)
    await watch_tickers(exchange_pro, symbols[:1])
    await exchange_pro.close()

if __name__ == '__main__':
    asyncio.run(main())

sc0Vu commented 1 week ago

@Crypto7816 the timestamp represents closing time for the 24h ticker (by default), and it's from exchange. I wonder whether that's a correct way to measure network latency.

Crypto7816 commented 1 week ago

@Crypto7816 the timestamp represents closing time for the 24h ticker (by default), I wonder whether it's a good way to measure network latency.

Could you please provide me with an appropriate method to measure latency?

ccxt / ccxt