PubSub Communication patterns.

[ronenhamias]

Motivation:

lets imagine we have 2 services in our cluster each service might go online and offline at any given point of time.

Service A depends on service B so we want to subscribe to this service when it appears in the cluster and wait for it to stream data to service B

currently this can be done using discovery events from service registry and service discovery.

to reduce the user boiler-plate we can offer an handler to service interface that will delegate events regards service dependencies in the cluster:

Proposed Solution

provide more way for user to react on discovery events by declaring handlers on service interface with regards to the service they want to "trap".

public interface MyService {

   @Discovery(MyService.class)
   void onDiscovery(Flux<DiscoveryEvent> events);

}

public class MyServiceImpl implements MyService, {
  Disposable disposable;
  @Override
  void onDiscovery(Flux<DiscoveryEvent> events){
     events.filter(DiscoveryEvent::isRegistered)
     .map(e -> 
     disposable = event.service().sayHello(someRequest).subscribe()
    ).subscribe()  
  }
}

Alternative approach:

call.discovery(MyService.class).subscribe(e->{
 events.filter(DiscoveryEvent::isRegistered)
     .map(e -> 
     disposable = event.service().sayHello(someRequest).subscribe()
    ).subscribe()  
}).

[ronenhamias]

Scalecube currently gives answer for the current pattern of communication between services

1. request response
2. request many
3. request channel.

after giving it some thought i think scalecube does not answer pubsub communication pattern. such as fanout, multicast, unicast and topic notion communication patterns between services.

having said that maybe we should consider Publisher / Consumer architecture for cases where we cant to broadcast messages to subscribers:

example scenario might be when we wish to broadcast stock symbols to multiple subscribers ect.

Although it can be done with Rsocket, i think Rsocket offered communication patterns mention above are less intuitive in such cases. as opposed for example to Aeron Consumer / Publishers. in addition current architecture of these communication patterns fits well when you deal with streams, but fits less well when you have to manage multiple subscribers in the cluster and emit one by one to each subscriber in order to create the effect of fanout for example as this will force you to write some boilerplate to manage retry and failures cases.

on the other hand Aeron publisher subscriber offer the notion of broadcasting which comes more intuitive when using UDP transport in these cases the need to manage logical streams on-top of single connections seems less needed thus the value of RSocket is questionable.

Maybe we should consider to offer PubSub layer on-top of Aeron transport with correlation to scalecube cluster to manage coordination of nodes in the cluster with microservices distributed architecture.

in such case we can offer Another transport and client similar to what we currently have with ServiceCall and RSocket patterns.

in case of PubSub patterns we can provide alternative channels for broadcasting messages and managing discovery and failures in the cluster on top of low latency high volume transport such as Aeron.