Does UTransport::register_listener match on source or sink address?

[sophokles73]

The RegisterListener() spec does not explicitly define the (type of) address to match on.

Register a UListener to receive message(s) for a given UUri (topic).

seems to suggest that the given topic should be matched on a message's destination address (sink attribute). However, that would mean that publish messages cannot be matched because they only contain an origin address (source attribute).

[tamarafischer]

Thanks for sharing @tamarafischer. The concern I have with your proposal is the matching rules you mentioned are fixed by the protocol meaning there is no value to expose this to applications and ask them to do the listener filtering, it should be inside of the core language libraries (not even in the transport implementations). We know that a publish, notification, request, response requires different filters and the rules are fixed, perhaps application might want additional filters but the base filters should be coded in the library.

so like this?

/**
 * Implementation of a UListenerWithMatcher that matches on messages from a source or part of a source.
 * The matching is done by comparing the configured uAuthority, uEntity and uResource, or parts of them if they are partial,
 * with the source of the message.
 * If the configured uAuthority, uEntity and uResource are partially build, only the configured parts are matched with the
 * incoming message, allowing for listener implementations to simulate wildcard matching.
 */
public interface MessageSourceUListener extends MessageUUriUListener {

    /**
     * The matching is done by comparing the configured uAuthority, uEntity and uResource, or parts of them if they are partial,
     * with the source of the incoming message.
     * @param uAuthority The configured {@link UAuthority} to match with the incoming message source {@link UAuthority}.
     * @param uEntity The configured {@link UEntity} to match with the incoming message source {@link UEntity}.
     * @param uResource The configured {@link UResource} to match with the incoming message source {@link UResource}.
     * @param messageSource {@link UUri} of the incoming message source.
     * @return Returns true if there is a match by comparing the configured uAuthority, uEntity and uResource, or parts of them if they are partial,
     *      with the source of the message.
     */
    default boolean matchSource(UAuthority uAuthority, UEntity uEntity, UResource uResource, UUri messageSource) {
        return matchWithUUri(uAuthority, uEntity, uResource, messageSource);
    }

}

/**
 * Implementation of a UListenerWithMatcher that matches on messages from a sink or part of a sink.
 * The matching is done by comparing the configured uAuthority, uEntity and uResource, or parts of them if they are partial,
 * with the sink of the message.
 * If the configured uAuthority, uEntity and uResource are partially build, only the configured parts are matched with the
 * incoming message, allowing for listener implementations to simulate wildcard matching.
 */
public interface MessageSinkUListener extends MessageUUriUListener {

    /**
     * The matching is done by comparing the configured uAuthority, uEntity and uResource, or parts of them if they are partial,
     * with the sink of the incoming message.
     * @param uAuthority The configured {@link UAuthority} to match with the incoming message source {@link UAuthority}.
     * @param uEntity The configured {@link UEntity} to match with the incoming message source {@link UEntity}.
     * @param uResource The configured {@link UResource} to match with the incoming message source {@link UResource}.
     * @param messageAttributes {@link UAttributes} of the incoming message.
     * @return Returns true if there is a match by comparing the configured uAuthority, uEntity and uResource, or parts of them if they are partial,
     *      with the source of the message.
     */
    default boolean matchSink(UAuthority uAuthority, UEntity uEntity, UResource uResource, UAttributes messageAttributes) {
        return messageAttributes.hasSink()
                && matchWithUUri(uAuthority, uEntity, uResource, messageAttributes.getSink());
    }

}

/**
 * Implementation of a UListenerWithMatcher that matches on a specific uProtocol Notification.
 * The type is a published message, with a sink that matches the listeners configured notification sink and a message source
 * that matches the listeners configured uAuthority, uEntity and uResource.
 */
public interface NotificationFromSourceUListener extends MessageSourceUListener, MessageSinkUListener, MessageTypeUListener {

    String NOTIFICATION_MESSAGE_TYPE = UMessageType.UMESSAGE_TYPE_PUBLISH.name();

    /**
     * Configure the type for the {@link MessageTypeUListener}
     * @return returns the type for the {@link MessageTypeUListener} to match.
     */
    @Override
    default String getType() {
        return NOTIFICATION_MESSAGE_TYPE;
    }

    /**
     * Enable filtering out notifications that are not targeted at a specific entity.
     * In most cases this is the uEntity describing the current application.
     * @return returns the {@link UEntity} that this notification is meant to reach.
     */
    UEntity getNotificationSinkUEntity();

    /**
     * This implementation of isMatch checks if the type in the UAttributes is the same as the type specified in the UListener, a published message,
     * and checks the sink matches the configured notification sink and the source of the message comes from the configured
     * uAuthority, uEntity and uResource.
     * This match enables matching notification messages that are pub.v1 with a specific sink, from specified sources.
     * Implementing isMatch enables the UListener to use its internal data to indicate to the transport layer if this
     * listener should be executed when a message is received.
     *
     * @param messageSource     The {@link UUri} of the message source.
     * @param messageAttributes The {@link UAttributes} of the message.
     * @return returns true if the UListener should be applied by the transport layer.
     */
    @Override
    default boolean isMatch(UUri messageSource, UAttributes messageAttributes) {
        return matchByType(getType(), messageAttributes.getType().name())
                && matchSource(getUAuthority(), getUEntity(), getUResource(), messageSource)
                && matchSink(UAuthority.getDefaultInstance(), getNotificationSinkUEntity(), UResource.getDefaultInstance(), messageAttributes);
    }
}

[tamarafischer]

Do you think you can implement ustreamer with mode 1? Or maybe we still need to define special UUri in mode 2 to listen to the specific UAuthority.

I am pretty sure that it can be implemented using mode 1 as well. The question is: do we want to? I tried to point out some advantages of mode 2 over mode 1 when running in-vehicle. Maybe @PLeVasseur, @stevenhartley and/or @Mallets have some opinion on whether this is actually relevant in the Zenoh context?

IMHO RegisteredListeners can be implemented using Zenoh, taking full advantage of Zenoh channels and all the other magic Zenoh provides. We don't need to implement the matching in the UListener since Zenoh will do that for me, or the MQTT broker will do that for me. All I need to do is implement the RegisteredListeners container interface - give it the tools to do the job being Zenoh or MQTT - uTransport can then easily change with other implementations of optimized containers for finding listeners - which by the way was the problem statement. That said, next week I will try coding this up, giving it a try. Then I can have a better understanding of the disadvantages. We have an end to end test that works, so we should be able to refactor without changing anything on the client side apart from the implementation of the UTransport interface with an object that uses Zenoh and a RegisteredListeners container implementation.

I guess the best way for me to understand if a design works well is code it up.

[PLeVasseur]

Good discussion. Trying to catch up :slightly_smiling_face:

Do you think you can implement ustreamer with mode 1? Or maybe we still need to define special UUri in mode 2 to listen to the specific UAuthority.

I am pretty sure that it can be implemented using mode 1 as well. The question is: do we want to? I tried to point out some advantages of mode 2 over mode 1 when running in-vehicle. Maybe @PLeVasseur, @stevenhartley and/or @Mallets have some opinion on whether this is actually relevant in the Zenoh context?

IMHO for uStreamer in the vehicle, it makes sense to lean on the protocol to perform the filtering as y'all are pointing out.

So if we go with option 2 for in the vehicle, we'd have to call all of registerRequest|Response|Publish|NotificationListener within the uStreamer for each UAuthority we're interested in, with the special UUri containing only UAUthority, right? @evshary, @sophokles73 for sanity checking

So in code, we'd have to:

let all_msg_listener: UListener = /* omitting */;
let up_client_foo = UpClientFoo::new();
let uuri_with_uauthority =  UUri {
                                 authority: Some(UAuthority{
                                    name: <doesnt-matter>,
                                    number: <some-specific-ip>
                                 }),
                                 ..Default::default()
                             };
let uuri_with_uauthority_2 = /* omitting */;
// ...
let uuri_with_uauthority_n = /* omitting */;

and then for each transport (here just foo is shown), we'd call the following 4 x methods for each UAuthority we want to listen for messages from:

up_client_foo.register_request_listener(uuri_with_uauthority, all_msg_listener);
up_client_foo.register_response_listener(uuri_with_uauthority, all_msg_listener);
up_client_foo.register_publish_listener(uuri_with_uauthority, all_msg_listener);
up_client_foo.register_notification_listener(uuri_with_uauthority, all_msg_listener);
// ...
up_client_foo.register_request_listener(uuri_with_uauthority_n, all_msg_listener);
up_client_foo.register_response_listener(uuri_with_uauthority_n, all_msg_listener);
up_client_foo.register_publish_listener(uuri_with_uauthority_n, all_msg_listener);
up_client_foo.register_notification_listener(uuri_with_uauthority_n, all_msg_listener);

If the ustreamer is mainly for in-vehicle scenarios, I agree with you. Zenoh can help filter other uninterested topics to improve performance. Then we should still keep the special UUri for this case.

Yes, I believe we'll still need the concept of the "special" UUri containing only a UAuthority for option 2.

[PLeVasseur]

I guess the best way for me to understand if a design works well is code it up.

@tamarafischer -- that'd be awesome to see. I'm trying to follow along with this thread of the conversation, but struggling a bit to piece it all together.

It'd be great to see an example that uses some of the pieces you outlined above like MessageSourceUListener and so on, and how that fits in with the concept you have for being able to write matching logic more generically.

[tamarafischer]

I guess the best way for me to understand if a design works well is code it up.

@tamarafischer -- that'd be awesome to see. I'm trying to follow along with this thread of the conversation, but struggling a bit to piece it all together.

It'd be great to see an example that uses some of the pieces you outlined above like MessageSourceUListener and so on, and how that fits in with the concept you have for being able to write matching logic more generically.

@PLeVasseur , I messed around with the deigns a bit this morning.

In the case of uTransport for HTTP or Azure EventHubs we have to do the matching ourselves since the actual transport mechanism does not contain this functionality - such as HTTP, or the solution does not scale such as having a topic for every vehicle subscription from a cloud application using a uTransport for Azure EventHubs. In these cases we have to have a way for an application developer to specify what messages he is interested in, along with the logic for handling these messages. The design pattern scales and is easy to test and the interfaces are very flexible.

If the vehicle needs a uTranport for HTTP, we are probably going to need a custom matching solution.

In the case of uTransport for MQTT in vehicle we probably do not have the scale issue that the cloud experiences, and we can delegate and optimize by using the MQTT topics directly to match a subscribed UURI to a UListener. I think the same goes for Zenoh that has the same concept as message brokers where you can specify a specific topic - or even wildcards or multiple topics, to be handled by application UListeners.

Both MQTT and Zenoh have the behavior of a message broker where the matching is pre-defined using a topic or topic wildcard. That said, this pre-defined logic is very optimized, and we probably want to enable these kinds of optimizations as well.

When I implemented the RegisteredListeners container for MQTT or Zenoh, I found that the containers held a lot of MQTT handlers or Zenoh Handlers. This led me to want to push the logic back into UTransport part since the matching was performed by the underlying transport layer and those details were leaking into the container logic - basically I lost my easy unit testing.

To summarize we have 2 use cases:

1. A uTransport that needs external business logic that developers need to provide to handle specific messages - custom matching.
2. A uTransport that is clever and optimized to matching UListeners to a pre-defined matching mechanism by UURI/Topic

I agree with @sophokles73's comment of "do we want to". Does it make sense to have one design patter that works for both use cases? What are the tradeoffs? What do we gain? For one thing, we did not gain flexibility or testability which were the reasons that I wanted the custom matchers.

It would be amazing if this could be pluggable. If a uTransport for MQTT in the car can optimize on pre-defined topic matching logic and a uTransport for MQTT in the cloud could optimize by using custom matchers and a single MQTT topic (or a couple of MQTT topics) with custom message matching.

These are the results of my morning exploration.

@stevenhartley , thoughts?

[stevenhartley]

In the the spirit of the IETF mantra "We reject kings, presidents and voting. We believe in rough consensus and running code” I believe we have rough consensus.

That being said, next steps are coding up proposal in up-java and up-rust (@tamarafischer and @sophokles73 will need your help) and then we will copy to the other languages and update the utransport specification. Stay tuned.

[stevenhartley]

@evshary, @MelamudMichael , @sophokles73 , @PLeVasseur , @tamarafischer, I want to share with you folks the proposal from Tamara that solves both use cases without having to add any new APIs!!! https://github.com/eclipse-uprotocol/up-java/pull/93.

The idea is simple. We use the marker pattern to extend the UListener interface per type of UMessage, then when someone calls registerListener() the implementation knows how to handle it (or not if it doesn't care) based on the marker using instanceOf(). The assumption then is that if they want to register a publish listener, they create a PublishUListener vs NotificationUListener.

Benefits of this proposal:

1. No changes to uTransport interface (we simply extend UListener)
2. We can deprecate RpcServer interface (need to handle the optimization we did in up-client-android-java with completableFuture outside of the transport API)
3. There is no stopping us from extending UListener in the future for more protocol types and/or external custom pattern mapping (if we wanted to)

For streamer, there are 2 options (I prefer option 1 because have to handle message types is transport specific anyways):

1. Pass base class UListener and UUri with UAuthority only, UTransports impl. has to handle any message type specific logic (i.e. zenoh).
2. Force uStreamer to register a listener for each type of message.

[tamarafischer]

Basically, the API is register a listener. UListener is an interface, and we can use domain modeling tools to define all types of interfaces.

If we need to add more information to the simple basic listener we can use marker interfaces that extend UListener or interfaces that make the developer add more information, such as UListenerWithMatchers.

IMHO, the important part is, we are still doing the one thing we are supposed to be doing - adding a listener.

A UTransport is now free to let developers know that additional information can be added using specific UListener implementations or the UListener will guess the intention (maybe leading to bugs).

This way of coding leaves a ton of wiggle room for changes and optimizations while letting the platform optimize and developers need to change one line of code, or developers have the space to add information for more control over what the platform is doing, such as adding custom matchers.

win-win, keep everybody happy.

[MelamudMichael]

I am fine with this

[sophokles73]

The UTransport interface is supposed to provide some means of letting client applications use a transport protocol without knowing about the transport protocol, i.e. separating the contract from the implementation. In a contract, all methods need to have clearly defined semantics. FMPOV that is the problem that originally wanted to solve because so far, registerListener has no clearly defined semantics, i.e. it is unclear to client code, what kind of messages will be passed to the listener being registered.

I guess the proposed approach can be made to work. However, I see the following issues:

1. Client code will still need to be able to register listeners for Publish, Notification, Request and Response messages. This means that every implementation of UTransport.registerListener actually MUST support the corresponding UListener (sub-)types along with their specific semantics. However, the method signature seems to indicate a dependency on UListener only when in reality, all implementations MUST depend on UListener as well as the (standard) subtypes in order to maintain interoperability across different UTransport implementations. With the proposed approach, this (hidden) dependency can only be expressed by means of the API documentation. Having dedicated methods for registering the listeners for specific message types would make this explicit in the API itself. It would also remove the necessity for declaring different marker interfaces.
2. Instead of requiring a specific UTransport implementations to extend the semantics of the standard registerListerner method and add more API documentation describing what kind of additional UListeners it supports, it would feel much more natural to me to simply extend the UTransport interface with an additional method for registering a transport specific listener, if necessary/desired. For client code this wouldn't make a difference because in any case, the client code would need to know about the specific transport implementation, either by creating the transport specific UListener subtype or by invoking a transport specific method for registering the (standard) UListener.

If the only goal would be to only have a single registerListener method in the UTransport interface then I guess the proposed approach could be made to work. However, FMPOV the more important goal is to clearly define the semantics of registering standard listeners while still allowing implementations to support custom listeners etc (the wiggle room).

FMPOV the former should be established by means of dedicated methods for registering Publish, Notification, Request, Response and All inbound Messages listeners. This way, the semantics and expectations towards implementers can be made very explicit while client code can always depend on this functionality to be available.

The latter could be achieved by simply extending (and documenting) the UTransport interface accordingly. Note, that in any case, client code would become dependent on the particular UTransport implementation as soon as it starts registering non-standard listeners.

[MelamudMichael]

Hi @sophokles73

I Prefer to have one API as it is simpler and clearer , and the user needs to have the understanding what type of message he is going to receive , it does not matter if it is explicit API or providing the correct interface type

Also i dont see an issue that the RegisterListener implementation will need to handle internally with different types of listeners, if a user will provide an unsupported listener , the API will fail.

[sophokles73]

Also i dont see an issue that the RegisterListener implementation will need to handle internally with different types of listeners, if a user will provide an unsupported listener , the API will fail.

I agree. However, the problem is, that it will fail during runtime only as the following example illustrates:

classDiagram
class UListener
<<interface>> UListener
class MySpecialListener
<<interface>> MySpecialListener
MySpecialListener --|> UListener
class UTransport
<<interface>> UTransport
UTransport: registerListener(UListener)
UTransport ..> UListener
class MySpecialTransport
MySpecialTransport ..|> UTransport
MySpecialTransport ..> MySpecialListener
class MqttTransport
MqttTransport ..|> UTransport

class Client
Client *-- MySpecialListener
Client o--> MySpecialTransport

This should work, if MySpecialTransport is implemented correctly:

var transport = new MySpecialTransport();
var listener = new MySpecialListener();
transport.registerListener(listener);

This will compile, but will not work as expected during runtime:

var transport = new MqttTransport();
var listener = new MySpecialListener();
transport.registerListener(listener);

On the other hand:

classDiagram

class UListener
<<interface>> UListener

class MySpecialListener
MySpecialListener ..|> UListener

class UTransport
<<interface>> UTransport
UTransport: registerPublishListener(UListener)
UTransport ..> UListener

class MqttTransport
MqttTransport ..|> UTransport

class MySpecialTransport
<<interface>> MySpecialTransport
MySpecialTransport --|> UTransport
MySpecialTransport: registerCustomListener(UListener)

class Client
Client *-- MySpecialListener
Client o--> MySpecialTransport

This will compile and work as expected

var transport = new MySpecialTransport();
var listener = new MySpecialListener();
transport.registerCustomListener(listener);

while this won't even compile

var transport = new MqttTransport();
var listener = new MySpecialListener();
transport.registerCustomListener(listener);

[tamarafischer]

With dependency injection and constructor injection developers should not be coding to the exact implementation of UTransport but to the interface.

In addition, pattern matching (Java is catching up) enables UTranport implementations to support the marker interfaces defined in the spec. There is no difference between specific custom methods, register event listener with a custom type parameter and spec defined UListener interfaces that need to be implemented anyway because developers need to implement onReceive.

UTransport implementations can ignore the custom interfaces and decide that they want to use the UURI and have whatever implementation magic to do the message routing.

Lastly, registerEventListener is NOT supported by all UTransport implementations - the cloud simply can't. That said, the mental model of UListener is still there - it is just wrapped in a container that is passed to the UListener at boostrap time and not all over the code base - registering and unregistering listeners all over the place. The flexibility of the container lets this be done in one place, allows for wild card matching and for easier testing.

Whatever is decided to be added to the spec or the client libraries is fine for me. Anyone is free to implement the best solution - if the open source community decides it has value taking it back, fine, if not, that is fine as well.

developers will always program to the UTransport interface - pluggable architecture If they don't, well that is another story altogether.

Kai, the drawng is a little different from the implementation and the dependencies are a little different - at least in the way I coded it up.

My two cents.

[PLeVasseur]

Couple things that stand out to me, that I wanted to highlight:

A UTransport is now free to let developers know that additional information can be added using specific UListener implementations or the UListener will guess the intention (maybe leading to bugs).

I'm in favor of designing the interface in such a way to minimize bugs. Could you give an example or two of how your proposal could lead to bugs for end-user code when someone is writing up a uE?

I agree. However, the problem is, that it will fail during runtime only [...]

I am leaning toward a design that can minimize, ideally eliminate, run-time bugs for our users.

I took a shot at writing this up in Rust over on this PR. There are certainly other ways of accomplishing what I think the design proposal is other than what I did, but those had various flavors of run-time failure points and I erred toward a design that tried to guide the user in the correct direction with interface design and pack as much failure into compile time as possible. The design does have some other weaknesses as well. I should go document some of these trade-offs.

[stevenhartley]

Folks, After many back & forth conversations and discussions, a compromise simplified proposal that avoids having to define functions per message type or inferring the message type from instanceof(UListener) (that doesn't work for all languages) is to add UMessageType to the regsiterListener() as shown below:

interface UTransport {
  UStatus registerListener(UUri, UListener, UMessageType);
}

Having said that, to @tamarafischer & @sophokles73 point, there is no stopping us from extending the interface for a specific implementation to provide external pattern matching as well, to that point I'll file another issue to address the Cloud use case so we can close this one. I will put forth a PR now with the above change to up-spec, if we cannot get general consensus on the PR, we will vote on Thursday.

BTW, as @PLeVasseur pointed out, NOTIFICATION was never a first class citizen UMessageType, it was aways inferred by PUBLISH + sink and this is not a good thing (legacy from CloudEvent days). As part of this effort we will finally add NOTIFICATION as a message type.

[PLeVasseur]

@stevenhartley -- so if we wanted to allow registerListener() to handle the 4 x standard message types (Publish, Notification, Request, Response) and also allow the possibility of say, collecting together a bunch of listeners and doing custom matching over all messages would we pass in MESSAGE_TYPE_UNSPECIFIED?

[tamarafischer]

A UTransport is now free to let developers know that additional information can be added using specific UListener implementations or the UListener will guess the intention (maybe leading to bugs).

Hmm, interesting question.

I think it will not be a problem. Transports such as MQTT and Zenoh have built in message routing according to topic - so you can't really add your own filtering - or it would be specific - say all published messages from body.access service Not sure how all notifications and rpc and published messages from body.access service would work.

On the other hand, the cloud can't support dynamic registering of events, hence the custom message matching has to be supplied - or use hard coded message handling in uTransport.

So, I am not sure both techniques will be used in the same application.

That said, not sure about a streamer that supports many transports.

I guess at the end of the day, from the perspective of an app developer or service developer it does not really matter. All should work. The grumbling of changing the interface yet again, should be ok since it is a small change hopefully in one place.

[evshary]

After taking a careful look at the comments above, I'm fine with no matter what is proposed by @tamarafischer or @stevenhartley It's a good design to add intentions to the listener by assigning UListener different types (Thanks for the Rust PoC from @PLeVasseur for my better understanding) However, as @stevenhartley pointed out, I'm not quite sure whether every language supports instanceof(UListener) or something similar. In this case, maybe UStatus registerListener(UUri, UListener, UMessageType); is a better way to generalize the usage.

[sophokles73]

if we wanted to allow registerListener() to handle the 4 x standard message types (Publish, Notification, Request, Response) and also allow the possibility of say, collecting together a bunch of listeners and doing custom matching over all messages would we pass in MESSAGE_TYPE_UNSPECIFIED?

IMHO that would be one option, but I agree with @stevenhartley 's earlier comment that this could be also provided by a method defined in a (cloud use case specific) extension of UTransport ...

[sophokles73]

After having given the whole thing some more thought, I now believe that my original question has put the focus on the wrong end of the message transfer. I was implying, that on the receiving end, a (pub/sub) transport implementation would actually be able to distinguish between source and sink when applying message filters. However, with an MQTT broker, the matching is simply done on the message's topic name using a topic filter (which can contain wildcards). With Zenoh, the matching is done in a similar fashion on the message's key, which also supports using wildcards.

I therefore believe that we should be able to get away with just the existing UTransport.registerListener(UUri, UListener) method for all cases, if we define some rules for how a UTransport maps uProtocol URIs to the underlying messaging infrastructure's addressing scheme.

Here's how the transfer of the different message types could be done by a UTransport implementation using an MQTT broker. The examples contain remote UUris only. However, the same principle applies to local addresses as well, using e.g, an empty string as the authority within the topic names/filters.

Transfer Publish message

1. A uEntity (origin-authority/origin-entity) creates a message with source: up://origin-authority/origin-entity/origin-version/origin-resource.instance#message sink: -
2. UTransport.send maps source to MQTT topic: origin-authority/origin-entity/origin-version/origin-resource/instance/message
3. Subscriber registers a listener for: up://origin-authority/origin-entity/origin-version/origin-resource.instance#message
4. which UTransport.registerListener maps to MQTT topic filter origin-authority/origin-entity/origin-version/origin-resource/instance/message

Transfer Notification message

1. A uEntity (origin-authority/origin-entity) creates a message with source: up://origin-authority/origin-entity/origin-version/origin-resource.instance sink: up://dest-authority/dest-entity/dest-version/dest-resource.instance
2. UTransport.send maps sink to MQTT topic: dest-authority/dest-entity/dest-version/dest-resource/instance/
3. Receiver (dest-authority/dest-entity) registers a listener for: up://dest-authority/dest-entity/dest-version/dest-resource.instance
4. which UTransport.registerListener maps to MQTT topic filter: dest-authority/dest-entity/dest-version/dest-resource/instance/

Transfer Request message

1. Service consumer (consumer-authority/consumer-entity) creates a message with source: up://consumer-authority/consumer-entity/consumer-version/rpc.response sink: up://provider-authority/provider-entity/provider-version/rpc.methodname
2. UTransport.send maps sink to MQTT topic: provider-authority/provider-entity/provider-version/rpc/methodname/
3. Service provider (provider-authority/provider-entity) registers a listener for: up://provider-authority/provider-entity/provider-version/rpc.methodname
4. which UTransport.registerListener maps to MQTT topic filter: provider-authority/provider-entity/provider-version/rpc/methodname/

Transfer Response message

1. Service provider (provider-authority/provider-entity) creates a message with source: up://provider-authority/provider-entity/provider-version/rpc.methodname sink: up://consumer-authority/consumer-entity/consumer-version/rpc.response
2. UTransport.send maps sink to MQTT topic: consumer-authority/consumer-entity/consumer-version/rpc/response/
3. Service consumer (consumer-authority/consumer-entity) registers a listener for: up://consumer-authority/consumer-entity/consumer-version/rpc.response
4. which UTransport.registerListener maps to MQTT topic filter: consumer-authority/consumer-entity/consumer-version/rpc/response/

Message routing

1. A streamer that is interested in all incoming messages for dest-authority registers a listener for: up://dest-authority////
2. which UTransport.registerListener maps to MQTT topic filter: dest-authority/+/+/+/+

I guess the same pattern can be applied using Zenoh as the transport, mapping the topics to Zenoh keys in a similar fashion and taking advantage of message filtering based on wildcards.

IMHO we need to explicitly define in the UTransport spec, which of the source/sink attributes needs to be used for addressing in the underlying messaging infrastructure per message type. In the UUri spec we should (or need to?) explicitly rule out using rpc as a resource name in Publish and Notification messages, I guess. In general, it seems necessary to be able to determine from a UUri whether it is used in a Publish/Notification, Request or Response message. For unresolved UUris this seems to already be the case due to the (clever) usage of the rpc.response resource name. For resolved UUris (which contain identifiers for entity and resource) we should explicitly define that identifier ranges of RPC resources MUST not overlap with the ranges used in Publish/Notification messages.

It also occurs to me now, that @tamarafischer (and probably others as well) might have been aware of this all the time already. If so, please accept my apologies for my ignorance.

Maybe @evshary and @PLeVasseur could check if I got it right and provide some feedback regarding the applicability in the Zenoh transport and the Rust Streamer? Maybe @devkelley could also cross check with the MQTT5 binding spec that is currently being created. @tamarafischer would this still be in line with the cloud use-case?

[tamarafischer]

@sophokles73 A Cloud implementation using MQTT would probably run into problems matching by specific UURI

A cloud application has to support millions of cars. Generating an MQTT topic for each subscription would be too much. In our implementations today, one MQTT Broker is not enough. We need more than one MQTT broker to service millions of cars (we use the scale unit design pattern). A cloud application is usually deployed - say using k8 - with more than one instance of the application (stateless architecture - or stateful with Akka) - but the main idea is that there is more than one node containing the application.

Registering a listener on one node does not mean that all the nodes are synched up with this registered listener and an event can come in on any node.

This is the main reason that cloud UTransports must have the UListener registered up front in the constructor. This is why, we are probably going to hard code the MQTT topics - say one for read, one for write, and maybe a high priority topic so we avoid the noisy neighbor pattern. The matching of events/messages to UListener becomes the responsibility of the application developer specifying the listener.

We tell them, you have one pipe (ok, 3 - but it is hidden), when you define a UListener, please tell me the business logic of how to match a UURI source, UAttiubutes uAttributes to a UListener.

In my code I have one for RPC, one for handling all subscription activity, and one for published events. Most use cases will have up to 5 of these matchers and code to handle the messages so it seems to be good enough.

The car is the most important since it uses really low level languages such as C that don't have the domain modeling tools we like. This is the main reason I think adding the type to the function signature is a good tradeoff.

We get explicit message handling - no guessing on the string of rpc.response (and others) Does not really affect the cloud since we are not calling register event listener method so no code change. The in-vehicle implementations might grumble a bit about the change, but it should be easy and in one place and code coverage should make sure bugs are ironed out.

The mental model of UListener is still the same. The registering of callback code to be called when messages arrive are still (mostly) the same.

So, yes, what you are suggesting works well for the cloud. Adding the addition type parameter works as well.

[stevenhartley]

@sophokles73 I still feel there is value however in deprecating RpcServer interface and using just the registerListener for all types of messages, do you agree?

[PLeVasseur]

I love your explanation by the way and the clarity with which you show the mapping. I think it'd be a great addition to the MQTT5 spec :slightly_smiling_face:

Maybe @evshary and @PLeVasseur could check if I got it right and provide some feedback regarding the applicability in the Zenoh transport and the Rust Streamer?

IMHO this seems doable from a Zenoh perspective, but I'll let @evshary add his much more informed opinion.

I opened up an issue on up-spec in which I try to nudge us toward documenting which UUri (source or sink) and how should be mapped for Zenoh in a similar manner to how you have in your comment.

[evshary]

Hi @sophokles73

Thank you for the detailed explanation. I think it matches the current upc-zenoh-rust implementation. What makes a difference is that Zenoh uses different mechanisms (query and queryable) to implement request & response. So, Zenoh heavily relies on the type definition of UUri. That said, which kind of type is the UUri: Publish, Notification, Request or Response?

Now I'm using the way mentioned here to discriminate the UUri type. Originally, I thought we could explicitly specify the type by having another argument, like UStatus registerListener(UUri, UListener, UMessageType); In that case, we didn't need to care about what UUri looks like. However, if we still keep the same API UStatus registerListener(UUri, UListener);, I agree with you we need to define the UUri carefully. Then everyone can tell at a glance which type of UUri it is.

As @PLeVasseur's request, let me describe how it works in Zenoh a little bit.

Notice the difference:

• We use Mircro Uri to generate Zenoh key.
• We need to map the registerListener to different mechanisms, that is, step 5.

Transfer Publish message

1. A uEntity (origin-authority/origin-entity) creates a message with

   • source: up://origin-authority/origin-entity/origin-version/origin-resource.instance#message
   • sink: -

2. UTransport.send maps source(Mirco Uri format) to Zenoh key

   • local (w/o origin-authority): upl/ + origin-resource-id + origin-entity-id + origin-version
   • remote (with origin-authority): upr/ + origin-authority-id/ + origin-resource-id + origin-entity-id + origin-version

3. Subscriber registers a listener for

   • up://origin-authority/origin-entity/origin-version/origin-resource.instance#message

4. which UTransport.registerListener maps to Zenoh key

   • local (w/o origin-authority): upl/ + origin-resource-id + origin-entity-id + origin-version
   • remote (with origin-authority): upr/ + origin-authority-id/ + origin-resource-id + origin-entity-id + origin-version

5. Since this is Publish URI, Zenoh subscriber will be used to bind the registerListener

Transfer Notification message

1. A uEntity (origin-authority/origin-entity) creates a message with

   • source: up://origin-authority/origin-entity/origin-version/origin-resource.instance
   • sink: up://dest-authority/dest-entity/dest-version/dest-resource.instance

2. UTransport.send maps sink to Zenoh key

   • local (w/o dest-authority): upl/ + dest-resource-id + dest-entity-id + dest-version
   • remote (with dest-authority): upr/ + dest-authority-id/ + dest-resource-id + dest-entity-id + dest-version

3. Receiver (dest-authority/dest-entity) registers a listener for

   • up://dest-authority/dest-entity/dest-version/dest-resource.instance

4. which UTransport.registerListener maps to Zenoh key

   • local (w/o dest-authority): upl/ + dest-resource-id + dest-entity-id + dest-version
   • remote (with dest-authority): upr/ + dest-authority-id/ + dest-resource-id + dest-entity-id + dest-version

5. Since this is Notification URI, Zenoh subscriber will be used to bind the registerListener

Transfer Request message

1. Service consumer (consumer-authority/consumer-entity) creates a message with

   • source: up://consumer-authority/consumer-entity/consumer-version/rpc.response
   • sink: up://provider-authority/provider-entity/provider-version/rpc.methodname

2. UTransport.send maps sink to Zenoh key

   • local (w/o provider-authority): upl/ + provider-resource-id(> [METHOD_ID_RANGE]) + provider-entity-id + provider-version
   • remote (with provider-authority): upr/ + provider-authority-id/ + provider-resource-id(> [METHOD_ID_RANGE]) + provider-entity-id + provider-version

3. Service provider (provider-authority/provider-entity) registers a listener for

   • up://provider-authority/provider-entity/provider-version/rpc.methodname

4. which UTransport.registerListener maps to Zenoh key

   • local (w/o provider-authority): upl/ + provider-resource-id(> [METHOD_ID_RANGE]) + provider-entity-id + provider-version
   • remote (with provider-authority): upr/ + provider-authority-id/ + provider-resource-id(> [METHOD_ID_RANGE]) + provider-entity-id + provider-version

5. Since this is Request URI, Zenoh queryable will be used to bind the registerListener

Transfer Response message

1. Service provider (provider-authority/provider-entity) creates a message with

   • source: up://provider-authority/provider-entity/provider-version/rpc.methodname
   • sink: up://consumer-authority/consumer-entity/consumer-version/rpc.response

2. UTransport.send maps sink to Zenoh key

   • local (w/o consumer-authority): upl/ + consumer-resource-id(which is 0) + consumer-entity-id + consumer-version
   • remote (with consumer-authority): upr/ + consumer-authority-id/ + consumer-resource-id(which is 0) + consumer-entity-id + consumer-version

3. Service consumer (consumer-authority/consumer-entity) registers a listener for

   • up://consumer-authority/consumer-entity/consumer-version/rpc.response

4. which UTransport.registerListener maps to Zenoh key

   • local (w/o consumer-authority): upl/ + consumer-resource-id(which is 0) + consumer-entity-id + consumer-version
   • remote (with consumer-authority): upr/ + consumer-authority-id/ + consumer-resource-id(which is 0) + consumer-entity-id + consumer-version

5. Since this is Response URI, the listener will be saved inside the UPClientZenoh and be called while receiving the response.

Message routing

1. A streamer that is interested in all incoming messages for dest-authority registers a listener for

   • up://dest-authority////

2. which UTransport.registerListener maps to Zenoh key

   • upr/ + dest-authority-id/ + **

TL;DR

IMHO, if we keep the original design, I suggested defining explicitly how different types of UUri should look like in up-spec. Both long UUri and micro UUri should be defined. Also, it would be better to provide API to tell the UUri type for convenience.

For example:

• Response: instance == "rpc.response"
• Request: instance == "rpc.methodname" and methodname shouldn't be "response"
• Publish: instance != "rpc.*" and message is not empty
• Notification: instance != "rpc.*" and message is empty

My two cents.

[sophokles73]

Based on some comments, I currently believe that a streamer will not be able to register a single listener with a UUri that covers all types of messages.

1. A streamer that is interested in all incoming messages for dest-authority registers a listener for

up://dest-authority////

2. which UTransport.registerListener maps to Zenoh key

upr/ + dest-authority-id/**

I guess this will not cover Publish messages which originate from other authorities but to which some uEntities on dest-authority have subscribed to. IMHO routing of Publish messages will need to be handled explicitly based on the notifications emitted by uSubscription service, i.e. a streamer will need to listen to these notifications and register dedicated listeners for subscribed topics, or am I mistaken?

@PLeVasseur @tamarafischer @evshary WDYT?

[sophokles73]

I also wonder about how a streamer would handle this case:

1. A service consumer (consumer-authority/consumer-entity) creates a message with source: up:/consumer-entity/consumer-version/rpc.response sink: up://dest-authority/dest-entity/dest-version/rpc.methodname

where dest-authority is != consumer-authority.

Does UTransport need to fill in consumer-authority before sending out the message? IMHO it should do so (or even has to?) because otherwise the service provider's repsonse message would be considered to be local to dest-authority (and not be routed back to consumer-authority, right?

@PLeVasseur thoughts?

[evshary]

I guess this will not cover Publish messages which originate from other authorities but to which some uEntities on dest-authority have subscribed to. IMHO routing of Publish messages will need to be handled explicitly based on the notifications emitted by uSubscription service, i.e. a streamer will need to listen to these notifications and register dedicated listeners for subscribed topics, or am I mistaken?

I guess we need a separate listener in ustreamer to listen to all Publish messages. Then, it will depend on whether we want the streamer to detect where the subscribers are. If not, then Publish message should be forwarded to all other authorities even if they don't care about the Publish message. If yes, then streamer needs to have a way to know who is interested in the Publish messages and then do the routing.

[tamarafischer]

Based on some comments, I currently believe that a streamer will not be able to register a single listener with a UUri that covers all types of messages.

1. A streamer that is interested in all incoming messages for dest-authority registers a listener for

up://dest-authority////

2. which UTransport.registerListener maps to Zenoh key upr/ + dest-authority-id/**

I guess this will not cover Publish messages which originate from other authorities but to which some uEntities on dest-authority have subscribed to. IMHO routing of Publish messages will need to be handled explicitly based on the notifications emitted by uSubscription service, i.e. a streamer will need to listen to these notifications and register dedicated listeners for subscribed topics, or am I mistaken?

@PLeVasseur @tamarafischer @evshary WDYT?

@sophokles73 I believe you are correct. In addition, IMHO, it provides more room for optimizations rather than a specific generic string.

[tamarafischer]

I guess this will not cover Publish messages which originate from other authorities but to which some uEntities on dest-authority have subscribed to. IMHO routing of Publish messages will need to be handled explicitly based on the notifications emitted by uSubscription service, i.e. a streamer will need to listen to these notifications and register dedicated listeners for subscribed topics, or am I mistaken?

I guess we need a separate listener in ustreamer to listen to all Publish messages. Then, it will depend on whether we want the streamer to detect where the subscribers are. If not, then Publish message should be forwarded to all other authorities even if they don't care about the Publish message. If yes, then streamer needs to have a way to know who is interested in the Publish messages and then do the routing.

IMHO, sending information out of a car when it is not needed might be an expense we don't want to pay for, hence optimizations would be a very good idea.

[tamarafischer]

In general messages should be immutable and middleware should not be changing them - this could open a whole nest of security issues. I am not sure I am following the exact problem, so just stating that in general, it is a good idea to work with immutable messages.

[PLeVasseur]

Based on some comments, I currently believe that a streamer will not be able to register a single listener with a UUri that covers all types of messages. [...] I guess this will not cover Publish messages which originate from other authorities but to which some uEntities on dest-authority have subscribed to. IMHO routing of Publish messages will need to be handled explicitly based on the notifications emitted by uSubscription service, i.e. a streamer will need to listen to these notifications and register dedicated listeners for subscribed topics, or am I mistaken?

@PLeVasseur @tamarafischer @evshary WDYT?

Yeah, this is an interesting point. As I mentioned in the uProtocol meeting earlier today, currently any Publish messages received would just be bridged onto every other transport the streamer is configured to use.

I think it's a good idea if @stevenhartley could take this into consideration with the streamer spec he's writing, on if the streamer would need to contain logic for knowing about subscrptions from uSubscription. From my point of view, it does make sense to implement something like this to cut down on the unnecessary chatter on the wire.

[sophokles73]

As I mentioned in the uProtocol meeting earlier today, currently any Publish messages received would just be bridged onto every other transport the streamer is configured to use.

I assume you are registering a listener for up:://origin-authority//// on the transport tapping into origin-authority in order to get a grip on all Publish messages originating from it, right?

[PLeVasseur]

I also wonder about how a streamer would handle this case:

1. A service consumer (consumer-authority/consumer-entity) creates a message with source: up:/consumer-entity/consumer-version/rpc.response sink: up://dest-authority/dest-entity/dest-version/rpc.methodname

Are the source and sink swapped in your example? I left a comment over on your new PR to up-core-api to help us clarify this point.

where dest-authority is != consumer-authority.

Does UTransport need to fill in consumer-authority before sending out the message? IMHO it should do so (or even has to?) because otherwise the service provider's repsonse message would be considered to be local to dest-authority (and not be routed back to consumer-authority, right?

@PLeVasseur thoughts?

Though I think source and sink are swapped, I'll roll with what you're doing in your example to not introduce any additional complexity.

Since the service consumer knows it has to go "off device", wouldn't it populate its authority? Or is it assumed that the service consumer knows it's contacting a service "off device", but wouldn't provide its own device's authority? If a service consumer contacting a service "off device" and is allowed to not include its own authority, gotta admit: I never pictured this scenario as a valid use case up till now.

If we go with the service consumer includes its authority (up://consumer-authority/consumer-entity/consumer-version/rpc.response) for getting services "off device", then the streamer wouldn't have to do anything special here. Just call UTransport::send() on the appropriate up-client-foo-rust.

[PLeVasseur]

I assume you are registering a listener for up:://origin-authority//// on the transport tapping into origin-authority in order to get a grip on all Publish messages originating from it, right?

Correct! :slightly_smiling_face:

[sophokles73]

Since the service consumer knows it has to go "off device", wouldn't it populate its authority? Or is it assumed that the service consumer knows it's contacting a service "off device", but wouldn't provide its own device's authority?

That is exactly what I would like to figure out :-) FMPOV we need to either explicitly specify that a service consumer MUST provide its authority if calling a method off-device or we need to specify that this it is UTransport::send's responsibility to fill in the sender's authority ...

[PLeVasseur]

Ah okay, I get it now. FMPOV it's the service consumer's responsibility to attach the authority of the device they are operating on if their are sending the Request off-device.

[sophokles73]

FMPOV it's the service consumer's responsibility to attach the authority of the device they are operating on

So an application developer would need to deal with this explicitly? Why not let the language lib do that for you?

[PLeVasseur]

If we can do it in the language lib, that's great. My intention was to say that IMHO if the Request is intended to go off-device, then it should have the authority stamped on both the sink and source UUris by the time it reaches the streamer.

[stevenhartley]

Merged change in spec and up-core-api

[sophokles73]

I assume you are registering a listener for up:://origin-authority//// on the transport tapping into origin-authority in order to get a grip on all Publish messages originating from it, right?

Correct! 🙂

This, however, will also give you all Notifications, RPC Requests and Responses originating from origin-authority. Are you manually filtering those out?

[PLeVasseur]

I assume you are registering a listener for up:://origin-authority//// on the transport tapping into origin-authority in order to get a grip on all Publish messages originating from it, right?

Correct! 🙂

This, however, will also give you all Notifications, RPC Requests and Responses originating from origin-authority. Are you manually filtering those out?

For those message types with sink UUris, the streamer would use the routing rules it's configured with.

Let's assume for the moment that we're talking about a message originating from the same device as the streamer is running on, which has:

• source authority: origin-authority
• sink authority: destination-authority
• type: Notification | Request | Response

If we get a Notification, Request, or Response like the above, then:

1. Check if we have a rule for routing from origin-authority onto destination-authority
2. If we do, lookup which Box<dyn UTransport> corresponds
3. Send on that Box<dyn UTransport>

Hope that made sense.

[sophokles73]

Does it matter, what device the streamer is running on? In any case, what I wrote before was not fully correct, as I realize now ;-)

If you register a listener for up://source-authority on a transport then you get

1. Publish messages originating from source-authority
2. Notifications destined to source-authority
3. RPC Requests destined to source-authority
4. RPC Responses destined to source-authority

And I guess you need to only forward the publish messages to the other transports and ignore the other messages, because those will be routed by the listener you register for the dest-authority, right?

[stevenhartley]

Reopening the issue as we need to address this

[stevenhartley]

All, as was discussed on the call this morning, we are going back to the basic principles of the uProtocol layers that are:

• uP-L1 Transport Layer: Interfaces and data model allows us to send/receive any form of message
• uP-L2 Communication Layer: Interfaces and data models to perform the RPC, Notification, and pub/sub communication patterns
• uP-L3 Application Layer: Business logic layer for defining the client/server interfaces and data model

Having said that uTransport needs the ability to send and receive messages of any kind but does not provide functionality for RPC, notification and pub/sub which means we need to update register_listener() to be able to match based on source and sink UUris and any communication specific APIs (ex. publish(), subscribe(), invokeMethod(), are done outside of uTransport.

[stevenhartley]

Closing based on #134