Transient AMQP publishers could cause stream writer tracking to grow too much

Every time a new stream publisher that uses the queue type interface is created it generates a unique writer id based on the pid and a guid. This is used for deduplication between the channel (queue_type) and the stream when the stream leader changes and the channel has unconfirmed messages pending.

The set of writer ids (inside the osiris writer) is held in memory and is only trimmed (to a maximum of 256 entries) when osiris rolls over to a new segments.

Workloads that use transient channels for publishing but rarely fill a stream segment could cause osiris writer heap memory (as the ids are less than 64 bytes) to grow and possibly also slow down the stream itself.

We improve this, either by trying to deregister then writer id on channel closure or continuously trim the writer id set inside osiris.

It is worth noting that osiris doesn't have an option to write a tombstone for the writer id even if it is explicitly removed and the osiris writer is restarted it will be recovered

rabbitmq / rabbitmq-server

Transient AMQP publishers could cause stream writer tracking to grow too much #7372