search

tardis-dev / serum-vial

Real-time WebSocket market data API for Serum
Mozilla Public License 2.0
173 stars 60 forks source link

Modify markets without restarting #33

Closed quazzuk closed 2 years ago

quazzuk commented 2 years ago

I was just wondering how difficult it would be to extend the rest interface to support adding/removing markets without restarting the feed and potentially causing gaps?

BTW Thanks for the great work, this is super useful!

thaaddeus commented 2 years ago

It would be possible I think, but would require some rework, perhaps another option would be to 'monitor' markets.json file and when it changes detect those changes and add/remove new markets, still it's not something on the roadmap right now.

quazzuk commented 2 years ago

Ok, thanks for letting me know. I'm looking for a way to recover orders and fills directly from the blockchain in the case of failure/restart. So far the best way I can come up with is running copies serum-vial in a redundant setup and pushing to a DB.

Do you happen to know a way of retrieving historical Request and Event Queue data from the blockchain in order to recover a given account's trade history?

Thanks

yamen commented 2 years ago

There is no built in way to look at event queue history - it's a circular buffer in a file (account) that's constantly overwritten. You can retrieve the event queue and parse it yourself rather than just relying on serum-vial processing it 'on change'. In fact a polling mechanism is likely more resilient (but less efficient) if you really must have absolutely every change. I've found RPC notifications only 99% reliable.

That said, you still have RPC availability to be concerned about, and I've found no solution that's 100% there either...

thaaddeus commented 2 years ago

@quazzuk latest version can boot quicker now if that helps in any way (it's 5 seconds now for my internal RPC node which does not rate limit and uses boo-delay: 50), there's boot-delay option you can use to tweak initial staggered delay when subscribing to the markets accounts data.