search

nats-io / nats-server

High-Performance server for NATS.io, the cloud and edge native messaging system.
https://nats.io
Apache License 2.0
16.04k stars 1.42k forks source link

Refactor delayed API responses #5948

Closed neilalexander closed 1 month ago

neilalexander commented 1 month ago

If we are going to dispatch delayed API errors as a part of the rate limiting work, then we need machinery that can send out delayed responses without leaking an endless amount of goroutines that are all just blocked on time.Sleep.

This refactors things so that the delayed API responses are queued into an IPQ and will be sent out by a single goroutine at the right time, which will be much friendlier on the scheduler.

Additionally we also add new sort functionality to the IPQs so that on each push, we can re-sort the pending items so that those that are due to be processed first will be popped first.

Signed-off-by: Neil Twigg neil@nats.io

derekcollison commented 1 month ago

I think we can just use this structure for responses or the hashed timed wheel I am working on. Not sure I would push ipq to do this.

https://pkg.go.dev/container/heap@go1.23.2

neilalexander commented 1 month ago

Closing in favour of #5955.