[Scylla] - Implement basic static rate limiting

[jr1221]

Description

Scylla JS has rate limiting. Turns out we need it for rust as well. It basically involves discarding any message that is sent within 100 ms (by default) of the last message per topic name.

Acceptance Criteria

• Can change the overall minimum time
• If done after #182, make this an atomic setting with an edit endpoint
• Performance hit is minimal

Proposed Solution

At the db handler layer introduce a HashMap<String, DateTime> or something which stores the time the last message came in. Do this only for messages that are not considered special types (i.e. messages we plan to batch insert). If the difference in time of a new message is smaller than the 100ms or whatever setting, discard the message and bail out of the processor. If it is greater store the new time in the hashmap and continue processing the data.

Mocks

No response

[jr1221]

@RChandler234 Do you want this to wait until after rust is merged or add it to #171 ?

[RChandler234]

I will look at the rust PR tn, and then you can merge it whenever you're cool with it, and then do this separate

[RChandler234]

@jr1221