[bitnami/nats] Jetstream Enabled and ReplicaCount 1 causes Jetstream to never start up

rblaine95 commented 1 month ago

Name and Version

bitnami/nats 8.3.5

What architecture are you using?

None

What steps will reproduce the bug?

$ kind create cluster
$ helm install nats \
    --set replicaCount=1 \
    --set jetstream.enabled=true \
    --set persistence.enabled=true \
    --set auth.enabled=false \
    oci://registry-1.docker.io/bitnamicharts/nats

$ kubectl port-forward svc/nats 4222

$ nats str ls
nats: error: could not list streams: JetStream system temporarily unavailable (10008)
$ nats kv ls
nats: error: JetStream system temporarily unavailable (10008)

The nats-0 pod will start up without issue, but Jetstream will never succeed in starting up, constantly warning about waiting for routes to be established:

[1] [INF] Starting JetStream cluster
[1] [INF] Creating JetStream metadata controller
[1] [INF] JetStream cluster recovering state
[1] [INF] Listening for client connections on 0.0.0.0:4222
[1] [INF] Server is ready
[1] [INF] Cluster name is nats
[1] [INF] Listening for route connections on 0.0.0.0:6222
[1] [ERR] Error trying to connect to route (attempt 1): dial tcp 10.96.167.20:6222: connect: connection refused
[1] [ERR] Error trying to connect to route (attempt 1): dial tcp 10.96.167.20:6222: connect: connection refused
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Healthcheck failed: "JetStream has not established contact with a meta leader"
[1] [WRN] Waiting for routing to be established...
[1] [INF] 10.96.167.20:6222 - rid:7 - Route connection created
[1] [INF] 10.244.1.1:64818 - rid:8 - Route connection created
[1] [INF] 10.96.167.20:6222 - rid:7 - Router connection closed: Duplicate Route
[1] [INF] 10.244.1.1:64818 - rid:8 - Router connection closed: Client Closed
[1] [INF] 10.96.167.20:6222 - rid:9 - Route connection created
[1] [INF] 10.244.1.1:14454 - rid:10 - Route connection created
[1] [INF] 10.96.167.20:6222 - rid:9 - Router connection closed: Duplicate Route
[1] [INF] 10.244.1.1:14454 - rid:10 - Router connection closed: Client Closed
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Healthcheck failed: "JetStream has not established contact with a meta leader"
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Healthcheck failed: "JetStream has not established contact with a meta leader"
[1] [WRN] Waiting for routing to be established...
[1] [WRN] Waiting for routing to be established...

Are you using any custom parameters or values?

replicaCount: 1
jetstream:
  enabled: true
persistence:
  enabled: true
auth:
  enabled: false

What is the expected behavior?

Jetstream should start up without issue

What do you see instead?

Jetstream never starts up unless replicaCount is greater than 1.

Additional information

Currently, the chart's behavior regarding clustering, replica count, and Jetstream isn't clearly defined. I propose the following improvements:

Clarify the relationship between clustering, replica count, and Jetstream.
Implement a more flexible configuration that maintains existing behavior while addressing edge cases.

Current behavior

If the cluster config is present, Jetstream requires a minimum of 2 replicas.
Omitting the cluster config allows replicaCount=1 and jetstream.enabled=true.

Proposed changes

Enable clustering if EITHER of these conditions is true:
- ReplicaCount > 1
- Jetstream is disabled

Implement this logic using the following Helm template condition:


{{- if or (gt (int .Values.replicaCount) 1) (not .Values.jetstream.enabled) }}
# Clustering definition
cluster {
{{- end }}

Otherwise, keep it simple and just {{- if gt (int .Values.replicaCount) 1 }}

rblaine95 commented 1 month ago

Omitting the cluster config if replicaCount < 2 results in Jetstream successfully starting up:

[1] [INF] Using configuration file: /opt/bitnami/nats/conf/nats-server.conf
[1] [INF] Starting http monitor on 0.0.0.0:8222
[1] [INF] Starting JetStream
[1] [INF]     _ ___ _____ ___ _____ ___ ___   _   __  __
[1] [INF]  _ | | __|_   _/ __|_   _| _ \ __| /_\ |  \/  |
[1] [INF] | || | _|  | | \__ \ | | |   / _| / _ \| |\/| |
[1] [INF]  \__/|___| |_| |___/ |_| |_|_\___/_/ \_\_|  |_|
[1] [INF] 
[1] [INF]          https://docs.nats.io/jetstream
[1] [INF] 
[1] [INF] ---------------- JETSTREAM ----------------
[1] [INF]   Max Memory:      122.07 MB
[1] [INF]   Max Storage:     10.00 GB
[1] [INF]   Store Directory: "/data/jetstream/jetstream"
[1] [INF] -------------------------------------------
[1] [INF] Listening for client connections on 0.0.0.0:4222
[1] [INF] Server is ready

javsalgar commented 1 month ago

Hi!

Thank you so much for the PR! The team will take a look

bitnami / charts