'408 Request Timeout' instead of '404 No Messages' from $JS.API.CONSUMER.MSG.NEXT.<stream>.<consumer> [v2.10.14]

[pablopla]

Observed behavior

Ephemeral consumer with 5 messages returns '408 Request Timeout' When sending

pub $JS.API.CONSUMER.MSG.NEXT.<stream>.<consumer> reply-to
{'batch': 10, 'no_wait': true}

Expected behavior

Response with status '404 No Messages' as documented.

Server and client version

nats-server: v2.10.14

Host environment

No response

Steps to reproduce

1. Create a stream.
2. Publish 5 messages to the stream.
3. Create ephemeral consumer.
4. Request a batch of 10 messages with no_wait:

   pub $JS.API.CONSUMER.MSG.NEXT.<stream>.<consumer> reply-to
   {'batch': 10, 'no_wait': true}

[ripienaar]

% nats req js.in.x "{{Count}}" --count 10
% nats sub -i --ack
% nats pub '$JS.API.CONSUMER.MSG.NEXT.X.PULL' --reply _INBOX.xTsW65ErOuzpA0blSfoKER.1EYmNa25 '{"batch": 20, "no_wait": true}'

I get the 10 available messages and then...

[#11] Received on "_INBOX.xTsW65ErOuzpA0blSfoKER.1EYmNa25"
Nats-Pending-Messages: 10
Nats-Pending-Bytes: 0
Status: 408
Description: Request Timeout

nil body

[Jarema]

quick check shows that we hit this condition instead of No Messages:

        // Check expiration.
        if (eos && wr.noWait && wr.d > 0) || (!wr.expires.IsZero() && now.After(wr.expires)) {
            hdr := fmt.Appendf(nil, "NATS/1.0 408 Request Timeout\r\n%s: %d\r\n%s: %d\r\n\r\n", JSPullRequestPendingMsgs, wr.n, JSPullRequestPendingBytes, wr.b)
            o.outq.send(newJSPubMsg(wr.reply, _EMPTY_, _EMPTY_, hdr, nil, nil, 0))
            wr = remove(pre, wr)
            continue
        }

[derekcollison]

We could adjust for 2.11 if we feel that its confusing.

[Jarema]

I'll take a look at this.

[ro0NL]

we experienced a similar issue

receive 404 dispatch a message receive 408 for a while receive 404

message lost

[Jarema]

@ro0NL This looks different. The original issue does not loose any messages. Can you provide more details, with client and server version?

[ro0NL]

nats  c sub
? Select a Stream some
? Select a Consumer some
--- subject: _INBOX.nnnV8WdA5BsPx0KEl7HcWX.gPUmUrsm

Headers:

  Nats-Pending-Bytes: 0
  Status: 408
  Description: Request Timeout
  Nats-Pending-Messages: 1

Data:

nats: error: could not Acknowledge message: nats: message does not have a reply

recreating the consumer solved it (nats c rm)

[ro0NL]

client version 0.1.4 server version 2.10.16

[Jarema]

@ro0NL Can you provide nats c info and nats s info for relevant stream and consumer?

[ro0NL]

$ nats s info some
Information for Stream some created 2024-04-30 04:43:55

              Subjects: some
              Replicas: 3
               Storage: File

Options:

             Retention: WorkQueue
       Acknowledgments: true
        Discard Policy: Old
      Duplicate Window: 10.00s
     Allows Msg Delete: false
          Allows Purge: true
        Allows Rollups: true

Limits:

      Maximum Messages: unlimited
   Maximum Per Subject: unlimited
         Maximum Bytes: unlimited
           Maximum Age: unlimited
  Maximum Message Size: unlimited
     Maximum Consumers: unlimited

Cluster Information:

                  Name: nats
                Leader: nats-2
               Replica: nats-0, current, seen 553ms ago
               Replica: nats-1, current, seen 553ms ago

State:

              Messages: 0
                 Bytes: 0 B
        First Sequence: 359
         Last Sequence: 358 @ 2024-07-16 09:04:33 UTC
      Active Consumers: 1

---

$ nats c info some some
Information for Consumer some > some created 2024-07-16T09:02:39Z

Configuration:

            Durable Name: some
               Pull Mode: true
          Deliver Policy: All
              Ack Policy: Explicit
                Ack Wait: 5m0s
           Replay Policy: Instant
      Maximum Deliveries: 1
       Max Waiting Pulls: 10,000

Cluster Information:

                    Name: nats
                  Leader: nats-2
                 Replica: nats-0, current, seen 404ms ago
                 Replica: nats-1, current, seen 404ms ago

State:

  Last Delivered Message: Consumer sequence: 2 Stream sequence: 358 Last delivery: 19m9s ago
    Acknowledgment Floor: Consumer sequence: 2 Stream sequence: 358 Last Ack: 16m34s ago
        Outstanding Acks: 0
    Redelivered Messages: 0
    Unprocessed Messages: 0
           Waiting Pulls: 0 of maximum 10,000