Abstract RabbitMQ away - Githubissues

Its 2017, we are 2 days away from the Java 9 release which features Flows (aka reactive streams), whereas other frameworks such as RxJava have been around for quite a while - and here we are requiring developers to invent and implement their own binary communication protocols and doing synchronization with semaphores.

There is nothing wrong with AMQP and byte channels, and its a way to achieve interoperability between heterogeneous systems. And its good that the platform has these use cases in mind. However, this technology should be there to ease complex tasks, not to make simple ones complex. Hence, a benchmark implementer should not be forced to fiddle on that level unless this is what he/she wants.

The essence of this issue is, that the platform already attempts to abstract away some complexity of the RabbitMQ channels with the Abstract*Component classes, such as with receiveCommand(), sendDataToTaskGenerator() or createContainer().

In regard to receiveCommand(): Callbacks and semaphores quickly get unmaintainable in complex systems. The primary reason is, that a callback is not an object that can be passed around where clients can subscribe to in order to encapsulate processing. For this reason, high level concepts such as Flows and CompletableFutures (aka Promises) were invented.

In regard to sendDataToTaskGenerator and createContainer, these should be abstracted with dependency injection, as in this concrete example, it would enable one to

Take control of the communication between components. The platform can of course by default provide RabbitMQ-based channel(factories). However, it would also be possible for a component to get a reactive, java-object based stream over a RabbitMQ channel injected.
Exchange the DockerServiceFactory with one that starts docker containers outside of the platform
Run benchmarks in arbitrary environments (also see #116)

class MyBenchmarkController {
  // The native RabbitMQ channel which backs any further abstraction
  @Resource("name=bc2tgRabbitMQ")
  protected Channel toTaskGenerator_A;

  // Java7 nio channels backed by RabbitMQ 
  @Resource("name=bc2tgJava7NIO")
  protected WriteableByteChannel toTaskGenerator_B;

  // Java 9 Flow backed by the RabbitMQ channel; gets rid of callback methods
  @Resource("name=commandQueue")
  protected Publisher<String> commandQueue;

  @Autowire
  protected DockerServiceFactory dockerServiceFactory;

  void init() {
    // Guava services: See https://github.com/google/guava/wiki/ServiceExplained
    Service dockerContainer = dockerServiceFactory.create(imageName);
    dockerContainer.start();
    ...
    dockerContainer.stop();

   commandQueue.subscribe(...);
  }
}

Of course, referencing a DockerServiceFactory directly in a component would be bad practice, as this would needlessly tie the component to docker and image names. Instead, a component should reference a ServiceFactory that may or may not be backed by the DockerServiceFactory.

hobbit-project / platform

Abstract RabbitMQ away #117