Add `consumer_limits` and `first_seq` to Stream Config

[Jarema]

Overview

Two additional fields were added in 2.10:

• consumer_limits
• first_seq

consumer_limits

A new field in StreamConfig that is a struct representing consumer limits created on given stream.

• It will not enforce limits on existing consumers. Having consumers with values higher than new limits (so it includes consumers with defaults) will result in error when setting the limits. No limits will be set at all.
• After setting new limits, when creating new consumers, their defaults will be lowered to the current stream consumer limits if those are lower than previous limits or lower than global nats-server defaults.
• If new consumers have explicitly set values for inactive_threshold or max_ack_pending, those will be left as they are. If passed value is too high (above limits), consumer creation will fail. Stream will not lower those values.

// In Stream Config
type StreamConfig struct {
       ...
    // The following defaults will apply to consumers when created against
    // this stream, unless overridden manually.
    ConsumerLimits StreamConsumerLimits `json:"consumer_limits"`
       ...
}

// Struct representing the `consumer_limits` field value.
type StreamConsumerLimits struct {
    InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
    MaxAckPending     int           `json:"max_ack_pending,omitempty"`
}

first_seq

Defines sequence from which stream starts when created

type StreamConfig struct {
       ...
    FirstSeq     uint64           `json:"first_seq,omitempty"`
       ...
}

Not documented in ADRs. Not mandatory for 2.10 client release if causes release delay.

Clients and Tools

• [x] Schemas @ripienaar https://github.com/nats-io/jsm.go/pull/477
• [x] CLI @ripienaar https://github.com/nats-io/natscli/pull/868
• [x] Terraform @ripienaar
• [x] Github Actions @ripienaar
• [x] Go @piotrpio
• [x] Java @scottf first_seq https://github.com/nats-io/nats.java/pull/962
• [x] Java @scottf consumer_limits https://github.com/nats-io/nats.java/pull/979
• [x] JavaScript @aricart first_seq pr - 7/27/23 consumer limits
• [x] .Net @scottf first_seq https://github.com/nats-io/nats.net/pull/813
• [x] .Net @scottf consumer_limits https://github.com/nats-io/nats.net/pull/818
• [ ] C @levb
• [ ] Python @wallyqs
• [ ] Ruby @wallyqs
• [x] Rust @Jarema @caspervonb

Other Tasks

• [ ] docs.nats.io updated @bruth
• [ ] Update ADR to Implemented
• [ ] Update client features spreadsheet

Client authors please update with your progress. If you open issues in your own repositories as a result of this request, please link them to this one by pasting the issue URL in a comment or main issue description.

[ripienaar]

@neilalexander maybe you can explain the behaviour of these for us, it says:

    // The following defaults will apply to consumers when created against
    // this stream, unless overridden manually.

But also these are limits?

So are they maximums or defaults or both a default and a maximum? I want to be sure to get the description in the schemas right

[Jarema]

I added some explanations based on our discussions.

[scottf]

I've talked directly to Neil and Tomasz. There is some clarity to be gained on the explanation of consumer limits, but I don't think there is anything for the client to do, the server will return an error if there is a constraint violation when updating a stream of adding/updating a consumer. Maybe the explanation just needs to be proliferated to the docs.