pub/sub message queue wss push updates

[raffis]

Is your feature request related to a problem? Please describe

Currently there is no possibility to push updates to clients.

V3 will integrate long polling #122 but this will be too resource intensive for various reasons if there are too many connections. Long polling is more resource intensive in general, MongoDB is not asynchronous in PHP (even with swoole #338 ).

Describe the solution you'd like

An async message broker should get implemented in the stack while resource changes via the rest api (or internal changes) will get pushed to the queue by the balloon server. V3 will implement swoole #338 as http server with an event loop and should also implement a websocket server. The websocket server loop should subscribe changes from the message queue and publish all resources to receivers.

The resource factories (which will get implemented in v3, with the concept from tubee) should determine what resources should get published to what users and this information must be stored in the queue as well. With that information the websocket loop can push those resources to the connected clients and determine what resources must be pushed to what clients.

Possible messages queues:

• redis (Has swoole support!)
• RabbitMQ (There are async clients for PHP around, not sure if they're compatible to swoole)
• MongoDB (As described bellow as alternative)
• Something bigger like kafka, but I don't see need for that and is probably a too heavy design and feels not lightweight at all.

A working implementation of this will also mean the end of the delta /api/v2/delta endpoint. However the delta endpoint will still be supported in the apiv1.

Note that long polling #122 will still get implemented, #365 can be seen as better alternative to long polling and should be preferred by most clients which require a persistent connection for as long as possible. Long polling might be better if one wants to watch updates of a single resource pool over a short period of time.

Describe alternatives you've considered

Using the existing MongoDB and the loop may watch over all collections. This however has several disadvantages:

• MongoDB implementation is not async in PHP, even not with swoole.
• What users see what resources is not stored in any resources directly.

An alternative possibility however could be to only subscribe to events and publish only those to clients. But not all resources (updates,create,delete) are currently represented as events.

Additional context

Add any other context or screenshots about the feature request here.

[raffis]

RFC-2 pending.

[raffis]

Might also look into sse. But probably not much different to long pulling.

[raffis]

Actually Using Swoole #338 and MongoDB does work. I've implemented a mongodb changestream within a swoole process which pushes events to connected clients (ws).