Market Data

[sneilan]

Create a separate market data process that clients connect to and recieve all market data. This could be a pub-sub redis server. I'm limiting scope of this feature to L1 market data (current bid / ask). Leaving L2/L3 on the roadmap.

[sneilan]

When Bid/Ask change, OrderBook will send updates to a ring buffer that clients can subscribe to for market data.

Considerations

• What happens if bid / ask is updated but not all clients have recieved previous bid / ask.
• What is the maximum number of clients?
• Should a fan-out architecture be used? Will that make the exchange harder to install?
• Should we send base64 encoded data instead of binary for the websocket addition?

Ideas

• At most once delivery. If client didn't recieve previous market data, don't bother sending if new market data came in.
• Maybe 10k maximum clients.
• Websockets can transfer binary data.

[sneilan]

For now, this will be a simple pub-sub architecture. OrderBook will publish market data updates to a ring buffer. Another SocketServer instance will run on a separate port, subscribe to this ring buffer and send each market update to all clients in order.

OrderBook -> Ring Buffer -> Separate MarketSocketServer -> Client(s)

[sneilan]

Market Data Ring Buf Data Structure

struct L1MarketData {
    unsigned char version = 0;
    char type; // 'b' for bid and 'a' for ask. Expandable to other data types.
    unsigned int val;
    unsigned long long time_ns;
};

Bid and ask can change separately so they are sent separately. If bid / ask null, send zero. However, if prices go negative like crude oil did during the pandemic, 0 for null could cause issues. Can fix in an updated version.

No monotonic counter in favor of time in nansoseconds. Possible for multiple market updates in same nanosecond but t does not matter as market data is an eventually consistent design.

Version added in case schema changes.