search

nats-io / nats-server

High-Performance server for NATS.io, the cloud and edge native messaging system.
https://nats.io
Apache License 2.0
16.01k stars 1.41k forks source link

Queue Groups on leaf clusters not balancing correctly when messages are routed in from hub cluster [v2.10.22] - Variant 2 #6040

Closed roeschter closed 3 weeks ago

roeschter commented 4 weeks ago

Observed behavior

Queue Groups on leaf clusters not balancing (only one subscriber receiving messages) in certain assymetrically connected leaf cluster

Expected behavior

Queue Groups always balance when all subs connection in the same leaf cluster

See also: https://github.com/nats-io/nats-server/issues/5972

Server and client version

Server 2.10.22 nat-cli from main

Host environment

Windows (Michael) - Linux (customer)

Steps to reproduce

Set up a hub cluster with nodes HUB1, HUB2, HUB3 Set up a leaf cluster with nodes LEAF1, LEAF2, LEAF3 Connect the leaf nodes as follows: (LEAF1 and LEAF2 connect to the SAME Hub - this is NOT a typo) LEAF1 → HUB1 LEAF2 → HUB1 LEAF3 → HUB3 Start queue group listeners on LEAF1 and LEAF2 nats --context LEAF1 sub --queue=q1 foo nats --context LEAF2 sub --queue=q1 foo

Publish to HUB3 nats --context HUB3 pub foo Hello

roeschter commented 4 weeks ago

Config 20241022_queue_group_issue_2.zip

kozlovic commented 3 weeks ago

@roeschter I believe I fixed the issue in #6043. If you want to give it a shot.

roeschter commented 3 weeks ago

Question from end user. Whats the approximate release of this in 2.10? (2.11.23?)