Subscribing with a topic of "" and prefix matching should subscribe to all topics

[RogMicrotest]

From my question at stackoverflow:

I'm trying to monitor all topics published on a crossbar.io broker. My attempt so far is to subscribe to topic="" with matching policy of 'prefix' but that doesn't seem to work, whereas topic="a" with matching policy of 'prefix' does see all events for topics starting with 'a'.

A proprietary broker does allow subscribing to all topics in this way, so I would expect crossbar.io to be the same?

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/40139190-subscribing-with-a-topic-of-and-prefix-matching-should-subscribe-to-all-topics?utm_campaign=plugin&utm_content=tracker%2F462544&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F462544&utm_medium=issues&utm_source=github).

[oberstet]

This should work .. needs investigation

[RogMicrotest]

It would be nice if it did work, however on re-reading the 'spec', it does state that a URI cannot be an empty string and hence using that as argument to the SUBSCRIBE is not valid. On the other hand, it would be nice if it worked...

[oberstet]

I have finally come around and analyze this - took me almost 2h. Turns out to be a race-like condition:

1) CB produces WAMP/CB meta events .. and these include events when a subscription is created here 2) when a client prefix subscribes to "", this also includes WAMP/CB meta events, and these events are dispatched to all eligible subscribers - including the one that currently subscribes

3. however, these events (destined for that currently subscribing client) are dispatched before the clients subscribe request returns! and hence, the client will receive an EVENT for a yet unknown subscription ID - this blows up here upon the client will drop the connection immediately

---

A similar situation (a real race) might happen even with normal subscribes, but actually I think because of the way Twisted or async programming works (single-threaded run-to-completion), and given our current code, this can't happen.

[oberstet]

Here is a log trace (with the blowing up in AB changed to logging at warn level):

2017-04-18T16:11:57+0200 [Container   31912] backend joined: SessionDetails(realm=<realm1>, session=1939439143236387, authid=<FY74-MTRN-W573-VNAR-Y9Y3-FURW>, authrole=<backend>, authmethod=anonymous, authprovider=static, authextra=None, resumed=None, resumable=None, resume_token=None)
2017-04-18T16:11:57+0200 [Container   31912] EVENT received for non-subscribed subscription ID 8461139667682785: event=Event(subscription=8461139667682785, publication=3002397316154480, args=[1939439143236387, {'id': 8461139667682785, 'created': '2017-04-18T14:11:57.105Z', 'uri': '', 'match': 'prefix'}], kwargs=None, publisher=1008179936101353, publisher_authid=None, publisher_authrole=trusted, topic=wamp.subscription.on_create, retained=None, enc_algo=None, enc_key=None, enc_serializer=None, payload=-), subscriptions={}
2017-04-18T16:11:57+0200 [Container   31912] EVENT received for non-subscribed subscription ID 8461139667682785: event=Event(subscription=8461139667682785, publication=8412533448959860, args=[1939439143236387, 8461139667682785], kwargs=None, publisher=1008179936101353, publisher_authid=None, publisher_authrole=trusted, topic=wamp.subscription.on_subscribe, retained=None, enc_algo=None, enc_key=None, enc_serializer=None, payload=-), subscriptions={}
2017-04-18T16:11:57+0200 [Container   31912] backend subscribed: Subscription(id=8461139667682785, is_active=True)
2017-04-18T16:11:57+0200 [Container   31912] event published: Publication(id=3454096723862343, was_encrypted=False)
2017-04-18T16:11:57+0200 [Container   31912] received event on topic1: counter is at 0
2017-04-18T16:11:57+0200 [Container   31912] details: EventDetails(subscription=Subscription(id=8461139667682785, is_active=True), publication=3454096723862343, publisher=1939439143236387, publisher_authid=FY74-MTRN-W573-VNAR-Y9Y3-FURW, publisher_authrole=backend, topic=<topic1>, retained=None, enc_algo=None)

[oberstet]

This runs even deeper: we do have a guard

if service_session and not subscription.uri.startswith(u'wamp.'):

around the publication of WAMP meta events (wamp.subscription.on_create and wamp.subscription.on_subscribe) which is supposed to prohibit publishing meta events for subscribes/registers on meta topics/procedures themselfes.

However, this guard does not catch cases when a pattern-based subscription on a topic not starting with wamp., but nevertheless matching wamp... is created.

This is exactly the case here, and it also explains why this meta event stuff was working at all: the examples we have all use subscribes like "wamp." not "".

[oberstet]

There are two approaches:

1. publishing meta events in the next reactor round (PR #1051 )
2. exclude the triggering session (using exclude option in publish)

Which one should we do?

[meejah]

Although sending the metaevent immediately to the new subscriber is changed behavior (arguably? I guess it didn't work at all before ;), it's not obvious to me that it's "wrong". That is, I could see some use-cases either way. If you're subscribing to "" it likely means you do want literally everything? (e.g. you want to audit what's subscribed to what?)

It seems ideal if the meta-event is processed via the include/exclude the same way as any other publish (as then subscribers who don't want to see it can exclude themselves?) I guess the default is to exclude the publishing session, so probably the "most consistent" behavior would be to not send the meta-event to the subscribing session unless they had exclude_me=False in their options.

[meejah]

Not sure if my last comment makes much sense, upon re-reading ... what I mean is: if you're subscribing to "" you're asking for all the events so it seems most -consistent to not do any filtering (and thus let even "your own" meta-event through).