nats-io / natscli

The NATS Command Line Interface
Apache License 2.0
453 stars 94 forks source link

Can't create pull consumer from cli #1056

Closed withinboredom closed 2 months ago

withinboredom commented 2 months ago

Observed behavior

Unless I'm just missing something obvious, it appears as though you cannot use the cli to create a pull consumer, as it fails with:

nats: error: Consumer creation failed: filtered consumer not unique on workqueue stream (10100)

Expected behavior

A pull consumer to be created.

Server and client version

server: v2.10.14 client: 0.1.4

Host environment

5 node cluster

Steps to reproduce

I first applied the stream:

# cat stream.json
{
        "name": "swytch",
        "subjects": ["swytch.>"],
        "retention": "workqueue",
        "storage": "file",
        "allow_rollup_hdrs": false,
        "deny_delete": true,
        "deny_purge": true,
        "num_replicas": 5
}
# nats str add --config stream.json
 # cat test.json
{
  "ack_policy": "explicit",
  "ack_wait": 300000000000,
  "durable_name": "swytch-activity",
  "filter_subject": "swytch.activity.>"
}
# nats consumer add --config test.json
nats: error: Consumer creation failed: filtered consumer not unique on workqueue stream (10100)

When created via an SDK with the exact same json, a consumer is created successfully.

ripienaar commented 2 months ago

The problem isn’t the pull consumer per se but the fact that work queue streams only allow one consumer to consume any subject.

So the error indicates you are trying to add a consumer that would clash with an existing one

withinboredom commented 2 months ago

the error indicates you are trying to add a consumer that would clash with an existing one

@ripienaar, what existing one? There are no consumers to clash with.

ripienaar commented 2 months ago

Show “nats s info” please

withinboredom commented 2 months ago
Information for Stream swytch created 2024-05-12 00:18:53

              Subjects: swytch.>
              Replicas: 5
               Storage: File

Options:

             Retention: WorkQueue
       Acknowledgments: true
        Discard Policy: Old
      Duplicate Window: 2m0s
     Allows Msg Delete: false
          Allows Purge: false
        Allows Rollups: false

Limits:

      Maximum Messages: unlimited
   Maximum Per Subject: unlimited
         Maximum Bytes: unlimited
           Maximum Age: unlimited
  Maximum Message Size: unlimited
     Maximum Consumers: unlimited

Cluster Information:

                  Name: nats
                Leader: nats-3
               Replica: nats-0, current, seen 889ms ago
               Replica: nats-1, current, seen 889ms ago
               Replica: nats-2, current, seen 889ms ago
               Replica: nats-4, current, seen 889ms ago

State:

              Messages: 0
                 Bytes: 0 B
        First Sequence: 0
         Last Sequence: 0
      Active Consumers: 0
            Alternates:         swytch: Cluster: nats
                        swytch_history: Cluster: nats
withinboredom commented 2 months ago

Hmm, maybe the mirror counts as a consumer, which is weird, because when created via SDK, it works fine.

Information for Stream swytch_history created 2024-05-11 23:44:01

              Replicas: 5
               Storage: File

Options:

             Retention: Limits
       Acknowledgments: true
        Discard Policy: Old
      Duplicate Window: 0s
            Direct Get: true
     Mirror Direct Get: true
     Allows Msg Delete: true
          Allows Purge: true
        Allows Rollups: true

Limits:

      Maximum Messages: unlimited
   Maximum Per Subject: unlimited
         Maximum Bytes: unlimited
           Maximum Age: 7d0h0m0s
  Maximum Message Size: unlimited
     Maximum Consumers: unlimited

Replication:

                Mirror: swytch

Cluster Information:

                  Name: nats
                Leader: nats-2
               Replica: nats-0, current, seen 498ms ago
               Replica: nats-1, current, seen 498ms ago
               Replica: nats-3, current, seen 498ms ago
               Replica: nats-4, current, seen 497ms ago

Mirror Information:

           Stream Name: swytch
                   Lag: 0
             Last Seen: 464ms

State:

              Messages: 0
                 Bytes: 0 B
        First Sequence: 0
         Last Sequence: 0
      Active Consumers: 0
ripienaar commented 2 months ago

Could be yeah. Making a mirror of a WQ is probably not really a good idea. Try without the mirror?

withinboredom commented 2 months ago

Yep, that was it

withinboredom commented 2 months ago

Adding the mirror after the consumers works fine via cli.

ripienaar commented 2 months ago

What server version?

withinboredom commented 2 months ago

It's in the initial issue report.