Investigate using STOMP

[thewilkybarkid]

We need to start writing the event bus spec soon. The result of the Walking Skeleton was to use RabbitMQ (AMQP 0.9.1 + extensions).

I've never been entirely comfortable with this as it's a specific implementation. While it's open source and easily runnable in a Docker container, it's still something we could have to run on AWS unlike our DBs, caches etc (unless another SAAS like CloudAMQP is used). AWS (and Azure) provide AMQP 1.0 services, which despite the name is very different to 0.9. RabbitMQ has a plugin that supports 1.0, but there seems to be little usage generally (I can't find a PHP client anywhere! The only Python client I can find is Qpid Proton).

AWS actually run ActiveMQ, which supports other protocols in the same way that RabbitMQ does. One of these is STOMP, which does have a decent amount of usage. It's quite a simple protocol that largely leaves it up to the broker to decide what to do with a message (there's no concept of queues, topics etc). So, I've started to experiment whether we can get it to support what we think we need in various implementations:

• Topics (messages sent to 0..n consumers, with wildcard routing/subscribing to multiple topics)
• Confirmations when sending/receiving
• Redelivery and dead lettering
• Durability (consumer can disconnect, reconnect and pick up messages sent during that time)
• Competing consumers

[thewilkybarkid]

Managed to get it roughly working with Stomp PHP against both RabbitMQ and ActiveMQ. Planning to get a repo up with examples of both.

[thewilkybarkid]

RabbitMQ was straightforward, having trouble getting competing consumers in ActiveMQ at the moment.

[thewilkybarkid]

Bit hidden in the docs, but VirtualTopics are the ActiveMQ solution.

[thewilkybarkid]

Experimentation in https://github.com/libero/stomp-event-bus-test.

[giorgiosironi]

RabbitMQ has heartbeats that may interact with the unacknowledged messages (if the client is still alive, don't resend messages to someone else). These are specified in STOMP but the client doesn't use them by default.

[giorgiosironi]

Competing consumers and filtering via routing key look reasonably simple. The provisioning side doesn't contain unnecessary information - it has queues and what is routed to them, but no knowledge about multiple clients like 1a and 1b.

[giorgiosironi]

There are indeed some strange protocol errors that are relevant to STOMP usage, also found:

consumer-1a_1  | 
consumer-1a_1  | In Version.php line 55:
consumer-1a_1  |                                                                                
consumer-1a_1  |   Unexpected response received. Expected a "CONNECTED" Frame to determine Ver  
consumer-1a_1  |   sion. Got a "MESSAGE" Frame!                                                 
consumer-1a_1  |                                                                                
consumer-1a_1  | 
consumer-1a_1  | consume
consumer-1a_1  | 

with ActiveMQ, in addition to the known RabbitMQ Subscription not found.

[giorgiosironi]

Also if we are working on proof of concepts trying out a Python client for sending/receive and interoperability is the natural next step. It would also clear up whether the protocol errors are a bug of the client library or something more serious.

[thewilkybarkid]

@GiancarloFusiello Could you add your notes from our conversation the other day?

[GiancarloFusiello]

@GiancarloFusiello Could you add your notes from our conversation the other day?

@thewilkybarkid I was thinking of adding a python service to your test project that produces messages to be consumed by one of the PHP consumers and consumes messages produced by one of the php services. How does that sound?

[thewilkybarkid]

Sounds great!

[GiancarloFusiello]

@thewilkybarkid FYI, I don't have permissions to push a PR to the stomp test project

[GiancarloFusiello]

STOMP doesn't seem well supported in the python community. Most of the better made libraries appear to have since been abandoned in favour of amqp libraries for Rabbitmq and kafka has it's own client. There is one library that appears to be "actively" supported, however, I found a small issue and, looking at the repo issues for the library, it appears the author is not answering issue requests as of mid 2018 or maintaining the project regularly.

Trying to get activemq to work how I would like seems very challenging and the project itself appears to be JMS focused. Rabbitmq is much easier to configure and has much better documentation.

I'm not sure if it's my unfamiliarity with STOMP but because it's basic it seems that you can only do basic messaging rather than use features such as an exchange/topic that fans out messages to multiple queues at the same time, for example.

[thewilkybarkid]

I'm not sure if it's my unfamiliarity with STOMP but because it's basic it seems that you can only do basic messaging rather than use features such as an exchange/topic that fans out messages to multiple queues at the same time, for example.

STOMP itself deliberately has no concept of different messaging patterns, but the Libero spec that we need to write can define them, and brokers configured appropriately.

Shame about the lack of existing Python support, but as clients are available in other languages (PHP, JS etc) and is well supported by two well-known brokers (that don't use custom protocols) I'm tempted to say this investigation can be marked as complete and successful.

[giorgiosironi]

So how does a Python application (should) use (one of) the two brokers? :thinking:

[thewilkybarkid]

Write code unfortunately. Can fork one of the existing libraries and improve/support it.

[thewilkybarkid]

Writing an RFC for this, will close when published.