nautechsystems / nautilus_trader

A high-performance algorithmic trading platform and event-driven backtester
https://nautilustrader.io
GNU Lesser General Public License v3.0
2.17k stars 488 forks source link

Port throttler for message bus to Rust #1904

Closed twitu closed 1 month ago

twitu commented 2 months ago

The throttler applies rate limits to messages passing being done in the single threaded engine. This is useful in ensuring that certain shared components can maintain rate limits across multiple strategies and that one component cannot overpower processing and finally as failsafe to prevent breaching rate limits of external systems.

https://github.com/nautechsystems/nautilus_trader/blob/e59971649ed90ffb518991bfa1fbf2f20ab7b986/nautilus_trader/common/component.pyx#L2655-L2678

Currently, it is used by the risk engine to rate limit messages sent to the execution engine. It works by checking the clock for elapsed time and whether rate limit has been exceeded. If exceeded, messages are throttled by being buffered or dropped otherwise it is sent. When messages are being throttled a time event is registered with the clock. When the event is triggered the throttler is "activated" and messages from the buffer are sent.

shapes at 24-09-02 20 25 33

This component needs to be ported to Rust to work with the newly designed message bus #1712 and data engine #1782.

As you can see from the diagram, the throttler is currently tightly coupled on the ingress side i.e. the component and the timer hold a reference to the throttler. The egress, output_send, output_drop is a custom implementation that is provided to the throttler, typically it passes the message on the message bus. In the ported Throttler, this design can be kept as is or made more generic by even receiving the inputs through the message bus.

Another important point to consider is that the throttler will not be exposed over pyo3, which means it can be made generic over its message type.

cjdsellers commented 2 months ago

Some history https://github.com/nautechsystems/nautilus_trader/issues/1202. @Pushkarm029 currently working on this.