RFC: Event bus - Githubissues

libero / community

A place for community-wide issues, discussion, resources, files and sharing

MIT License

0 stars 1 forks source link

RFC: Event bus #10

Open thewilkybarkid opened 6 years ago

thewilkybarkid commented 6 years ago

Problem

Libero needs to allow for disparate services to indirectly communicate with each other, so they can monitor and react to data changes.

Suggestion

Use RabbitMQ (AMQP 0.9.1 standard + publisher confirms extension) to allow for services to reliably emit notifications that something has happened
Define a standard for routing keys
- Namespacing
Services can act both in an upstream and downstream role
- Upstream services own and send messages to their ‘topic’ exchange(s)
- Downstream services bind their queue(s) to one or more exchanges
- Downstream services consume messages from their queue(s)

Concerns

RabbitMQ is an opinionated choice, but other existing options/standards don’t meet our requirements (AMQP has two flavours, and doesn’t a performant way of confirming publication)

giorgiosironi commented 6 years ago

Adding https://docs.google.com/document/d/1zTqwWQNOqrzVqmwnWUqKsHq8fHFLSByhTvMNrgBoIXI/edit here which briefly lists the open source alternatives to RabbitMQ that were evaluated (ZeroMQ and ActiveMQ), plus:

requirements to be met
the AMQP protocol and the RabbitMQ extensions necessary for fully reliable delivery
the driver and libraries support for PHP and Python, the main programming languages we have been using so far during prototyping

giorgiosironi commented 5 years ago

Pointing out this analysis was originally scoped to Libero Publisher, but is ~not~ now potentially a Libero-wide decision.

petereast commented 5 years ago

We've been doing quite a bit of work looking into a generic event bus system for libero-reviewer. Part of this work has been in collaboration with PubSweet where we tried to agree with Hindawi on the interface for a generic event bus, so we could support the "event bus" pattern without needing to align on the specific technology (as Hindawi use AWS SQS and SNS -- not the choice for libero). Reviewer currently includes an implementation of this pattern using RabbitMQ, details of which can be found here: https://github.com/libero/reviewer/tree/master/lib/event-bus

giorgiosironi commented 5 years ago

Would be interesting to include here whatever decisions are made at the PubSweet level, as these are a coupling point. For example if https://github.com/libero/reviewer/blob/master/lib/event-bus/src/event-bus/index.ts#L5-L17 goes into PubSweet and Libero products align themselves between each other, then Pubsweet dictates the format of messages for all Libero products.