search

AlignmentSystems / MatchingEngine

A toy implementation of a matching engine. Don't use this in the real world, it's a toy!
MIT License
3 stars 0 forks source link

Scaling-out #8

Closed JohnGreenan closed 9 months ago

JohnGreenan commented 10 months ago

At the moment the whole platform runs on one box. This is not viable for production use. The APIs between the components would need to be reworked inside the implementation, such that

the number of FIX engines can be increased and distributed to multiple boxes the number of OrderBooks can be increased and distributed to multiple boxes market data distribution can be performed from a specified set of UDP/Multicast ports

JohnGreenan commented 10 months ago

This is something that can be done relatively easily as a proof-of-concept.

Allow the Matching engine to interact with an OrderBook with a message oriented middleware queue in the middle rather than a Java ConcurrentLinkedinQueue.

In an enterprise implementation this would involve passing an SBE message around, for now this can use some naive Java serialisation/deserialisation model

JohnGreenan commented 9 months ago

Decided to implement using SBE rather than a naive Java serialisation/deserialisation model.

JohnGreenan commented 9 months ago

FIX>POJO>Binary to matching engine completed. OrderBook matches the order POJO event emitted by OrderBook. The POJO needs to be converted to Binary, published to Kafka and then collected by a process running alongside the FIX engine instance. Pattern suggestion - one topic per workflow item (execution report, unsolicited amendment, reject and so on) One thread per topic. Each topic listener converts binary to FIX and then queues up for the FIX engine to send to the counterparty.

JohnGreenan commented 9 months ago

resolved in #22