When is public/time called & other shenanigans

[nascarjake]

Hi there, I run a scaled app with many clients across a small group of IPs. For a long while now i have had intermittent issues with bybit banning my IPs. I have worked with their support on this issue over the course of nearly a year. It only seems to happen once in a while, and when it does its really hard to get it to calm down or stop forcing us to abandon an IP for a new one and obviously this looks terrible on my end.

All the issues stem from api access behavior. Specifically repetitive calls to /public/time I have confirmed with bybit that i never hit any of their rate limits, and they are solely upset with me over these repetitive calls which can reach up to 3.8 million calls in a few hours.

I use ccxt and Ive gone up and down with the developer over there trying to figure out why public/time would being called. There was actually at first an option I had on to sync time, which caused the endpoint to be called preceeding any call ccxt made to bybit. I have since turned this option off, and confirmed multiple times locally and on the production environment that the option is off. Where i did used to see evidence of ccxt calling that endpoint, I no longer do.

It just occurred to me today that I use this library for bybit websockets. Its the only websocket implementation i have right now, and its the only thing else that connects to bybit. I also strictly use this for websockets only. So I was wondering if perhaps this library calls public/time?

I have equipped xray trace logging just for this specific issue and I have found evidence that even today it is still calling public/time. On a server which is not banned, during normal service it doesnt seem to call it very often. On a server which is banned, its calling it constantly.

I also read in another thread here that someone else had an issue with sync time and that setting it to a really high timeframe (7 days for him) fixed the problem of it dropping. I also noticed in your reply you said the drops seem to happen at a certain time during the night. Im -5 UTC so idk if my night is the same as your night- but i have also noticed that if it does break once every few months it happens literally hours after i go to bed, like 3 or 4 am. Im a night owl and many times i just miss it. For a while i thought i was cursed lol.

Long story short I wanted to know when public/time is called and what can be done about limiting calls. I know theres a reconnection timeout - normally i would want it to reconnect fast if it just dropped like it sometimes does - but if it dropped and then had a public/time fail or something - i would want it to cool off or something.

Is there a way i can console out the public/time calls this library makes?

I do run multiple sockets, one or two per client, and then two for public calls as well. While we are on the subject, is there a limit i should be worried about here? I do load balance the wallets across servers but not with respect to any websocket worries at the moment.

Sorry for the long message, hope this provides enough information.

[nascarjake]

Also i just noticed that I have this in my package.json "bybit-api": "^2.1.3",

And when I look at your version history on github i only see 2.1.0 If i wasnt confused before, I sure am now.

And to make things even more confusing I ran npm ls bybit-api `-- bybit-api@2.1.5

Could this be part of the issue? Should i be locked to 2.1.0?

[nascarjake]

Also here is my websocket implementation - this object is passed to other classes who subscribe to what they handle, such as a PositionManager class. The code below is only the initialization of the user websockets.

class UserWebsocket {
    constructor(opts) {
        const wsOps = {
            key: opts.apiKey, 
            secret: opts.privateKey, 
            livenet: !opts.testnet,
        };

        this.clients = {
            testInverse: new WebsocketClient({...wsOps, market: 'inverse'}),
            testLinear: new WebsocketClient({...wsOps, linear: true, market: 'linear'}),
        }

        for(const [key, ws] of Object.entries(this.clients)) {
            ws.on('open', ({ wsKey, event }) => {
              log.info('connection open for websocket with ID: ' + wsKey + ' - ' + opts.apiKey);
              //console.log(event)
            });

            ws.on('response', response => {
              //log.info('user ws response ' + opts.apiKey, response);
            });

            ws.on('close', () => {
              log.verbose('connection closed ' + opts.apiKey);
            });

            ws.on('error', err => {
              log.error('user ws error ' + opts.apiKey, err);
            });
        }
    }
}

[tiagosiebler]

Hi Jake,

Thanks for putting all this together. The latest version is currently v2.2.2: https://github.com/tiagosiebler/bybit-api/blob/master/package.json#L3

If you're referring to the releases section of github, I should probably remove or automate that as I've unfortunately grown to neglect that part of the repo, in favour of npm releases & commit messages showing when I bumped the version number to trigger a release.

The sample you've posted is helpful in seeing an idea why you might be having this issue. Each WebsocketClient client creates an instance of the appropriate REST client: https://github.com/tiagosiebler/bybit-api/blob/master/src/websocket-client.ts#L257-L283

This is purely for the time sync feature, as the rest client can automatically try to estimate latency & differences between your system time and server time (although I'm not convinced in the effectiveness of this anymore, as you may see in my comments across my 3 currently public connectors).

Each rest client instance will (by default) start a timer through setInterval, which by default executes every 3600000 ms (3600 seconds or 60 minutes).

This timer is what will call the time endpoint at regular intervals. If you're spawning many WebsocketClient instances (but also discarding them, without an existing method to kill the timer), I would imagine these timers would keep running in the background, which could have a compounding effect in the number of time endpoint calls you're seeing. That depends though whether you only ever create one UserWebsocket instance for the life time of your process, or if you might create and attempt to discard those instances (unaware a timer may be left running).

What I would immediately suggest is having that timer turned off in your websocket clients. It's not essential and should work without. Something like this should do the trick to pass that flag to the rest client:

        const wsOps = {
            key: opts.apiKey, 
            secret: opts.privateKey, 
            livenet: !opts.testnet,
            restOptions: { disable_time_sync: true },
        };
        this.clients = {
            testInverse: new WebsocketClient({...wsOps, market: 'inverse'}),
            testLinear: new WebsocketClient({...wsOps, linear: true, market: 'linear'}),
        }

You can see the full options supported by the ws client here: https://github.com/tiagosiebler/bybit-api/blob/7c531161b5ba1cd447bfa035013a23bf1d262ffe/src/websocket-client.ts#L148

and this is the flag I'm suggesting: https://github.com/tiagosiebler/bybit-api/blob/7c531161b5ba1cd447bfa035013a23bf1d262ffe/src/util/requestUtils.ts#L9

which would prevent the timer from ever being created: https://github.com/tiagosiebler/bybit-api/blob/7c531161b5ba1cd447bfa035013a23bf1d262ffe/src/util/BaseRestClient.ts#L89-L92

there is one final section which could pose a problem in the IP ban scenario, where you might be attempting too-frequent reconnects, that's this line here: https://github.com/tiagosiebler/bybit-api/blob/7c531161b5ba1cd447bfa035013a23bf1d262ffe/src/websocket-client.ts#L477

This happens when trying to open an authenticated websocket connection. The client will try to fetch a time offset to reduce chances of any time-related issues in the signed message. It's currently not optional but I can add a flag to turn that off. Assuming your system time is in sync (e.g. using something like ntp in unix environments), this should still work perfectly fine without that offset.

[tiagosiebler]

Hi @nascarjake

Just pushed v2.3.0 with changes that should resolve your observations. Also sent you a message on telegram with other thoughts.