I've been working on a project to ingest L2 orderbook info. I'm mostly working with Binance and Kucoin and I know they allow a list of market id's to be able to be used for subscription. I had some issues with Binance using the single subscribe call with all their pairs. I would get 429's sometimes - due I think in the code to each orderbook snapshot being requested via rest.
I put Binance aside and focused on Kucoin. I created a subclass and monkey patched the subscription related functions. They have around 1500 pairs and I was able to break these out to chunks of 100 or less and then spawn multiple clients per each chunk. Each client called subscribe with a comma delimited list of those 100 pairs. My thinking behind this is it's more efficient since their api allows it. It's def a little more work on my side.
My question is has any of this been discussed or thought about before? I've used other non-Javascript websocket libraries for this type of thing with the comma list of pairs for subscribe and it's worked. Looking through the code as I've built my poc though, seems like there would be some substantial reworking since everything is based off just one pair per subscription call.
The project I'm currently working on needs efficient startup for sockets. I know the code has rate limiting but on Binance.us, it seemed something was going on (I'll table that for later, could be user error). I'm wondering if anyone has found that the single subscription call is ok? Or not? Thoughts in general on this?
Hello, thanks for this library.
I've been working on a project to ingest L2 orderbook info. I'm mostly working with Binance and Kucoin and I know they allow a list of market id's to be able to be used for subscription. I had some issues with Binance using the single subscribe call with all their pairs. I would get 429's sometimes - due I think in the code to each orderbook snapshot being requested via rest.
I put Binance aside and focused on Kucoin. I created a subclass and monkey patched the subscription related functions. They have around 1500 pairs and I was able to break these out to chunks of 100 or less and then spawn multiple clients per each chunk. Each client called subscribe with a comma delimited list of those 100 pairs. My thinking behind this is it's more efficient since their api allows it. It's def a little more work on my side.
My question is has any of this been discussed or thought about before? I've used other non-Javascript websocket libraries for this type of thing with the comma list of pairs for subscribe and it's worked. Looking through the code as I've built my poc though, seems like there would be some substantial reworking since everything is based off just one pair per subscription call.
The project I'm currently working on needs efficient startup for sockets. I know the code has rate limiting but on Binance.us, it seemed something was going on (I'll table that for later, could be user error). I'm wondering if anyone has found that the single subscription call is ok? Or not? Thoughts in general on this?
Thanks!