Improve support for consumer FilterSubjects

[wallyqs]

Given the following Stream setup with wildcards:

    js.AddStream(&nats.StreamConfig{
        Name:     "foo",
        Subjects: []string{"foo.*", "foo"},
    })

Using the JS context, currently there is no way to subscribe to set a filter subject that matches the wildcard and it has to match literally.

js.Subscribe("foo.1") // Stream lookup will fail since no stream matches `foo.1` literally
js.Subscribe("foo.*") // Works since stream lookup matches `foo.*`

As a workaround, it is needed to create a consumer that has the defined filter subject:

    // Create durable pull subscriber (no delivery subject)
    _, err = js.AddConsumer("foo", &nats.ConsumerConfig{
        Durable:       "foo",
        FilterSubject: "foo.1",
        AckPolicy:     nats.AckExplicitPolicy,
    })

Then, to subscribe it has to use BindStream and the durable name to disable the stream lookup:

sub, err := js.PullSubscribe("foo.1", "d-foo", nats.BindStream("foo"))

Ideally, we should be able to support the following as well:

// Avoid stream lookup, uses provided filter subject to create/attach to consumer
js.Subscribe("foo.1", "d-foo", nats.BindStream("foo")) 

// Does not set a filter subject and only binds to the stream (matches all subjects)
js.Subscribe("", "d-foo", nats.BindStream("foo"))

[derekcollison]

This should work, we match by subset wc matching algorithm to select the stream. If the filter subject is the subject passed into the js.Subscribe call should work, if it does not would want to understand why.

[ripienaar]

For JetSream I'd like to see a bit of a behavior change that relates to that.

Given a stream with subjects X.1 and X.2 I should be able to make a consumer that filters on X.* or X.>, in these cases it implies give me all the messages.

This seems to be the more obvious way this should work (we've seen repeated cases of users running into this and thinking its weird now) it also means that if I later edit the stream to also get Y.> and X.3 subjects my consumer will still be valid

I can open a JS issue if you like @derekcollison but wanted to just chip in here as this came up during the same converation that led to this change to nats.go

[derekcollison]

Not sure I agree there, subjects should be equal or a struct subset of the interest set for the stream.

[ripienaar]

Another case brought by a user, stream with A.A, A.B and C.A on it, there's no way to ask only for the A's with current behavior.

[derekcollison]

What is the enclosing subject for the stream itself? .?

If you want just messages with A, in that case, you need two consumers, A. and .A. Nothing to do with JetStream there.

[ripienaar]

The stream has exactly A.A, A.B and C.A as subjects, you can't in that case make a consumer for A.*

[derekcollison]

That is correct and by design tbh..

Although since this is a very specific JS consumer and not a wildcarded low level NATS subscription it could be possible since you will not have non-scoped messages every delivered by expressing something wider.. So maybe..

[ripienaar]

Yes, exactly my thinking, we should compare the wildcard against the stream and not the world since its a constrained view and based on several users running into this I think its the obvious thing people seem to expect.

This way my consumers can remain valid even through stream subject edits etc

[derekcollison]

I am onboard now..

[ripienaar]

I'll file a ticket :)

[wallyqs]

Multiple filter subjects is a feature from the server now so closing.