HTTP Events using HTTP Long Polling and pub/sub service

[thejhh]

Goal

Develop a stateless pub/sub event system for StatelessDB that supports multiple compute nodes without relying on third-party pub/sub services. This initial implementation aims to establish a simple and scalable event mechanism between StatelessDB services, laying the groundwork for future modular extensions.

We will utilize HTTP long polling as the method for event delivery due to its scalability, stateless nature, and ease of implementation over HTTP.

Architecture

1. Client Request:

   • The client sends an HTTP request to the StatelessDB cluster, including a timeout parameter (e.g., 30 seconds).

2. Server Processing:

   • The receiving compute server connects to the preceding StatelessDB compute node to listen for events related to the current request.
   • The server waits for events from the preceding node for up to the specified timeout period.
     • a. Event(s) In Queue: If some events are in queue, they will be instantly transferred.
     • b. Event Occurs while Waiting: If an event is received within the timeout, the server immediately responds to the client with the event information.
     • c. No Event Received After a Timeout: If no event occurs within the timeout, the server responds indicating that no events were received.
     • d. No More Events: Indicates there will not be more events and the subscription can be closed.

3. Response Structure:

   • The server's response includes details about the received event or indicates the absence of events.
   • The response's private section contains metadata about the event source or handling process.

4. Connection Management:

   • After responding, the compute server maintains the connection to the preceding compute node to listen for future events without immediately unsubscribing.
   • The server keeps the pub/sub connection active until one of the following conditions is met:
     • a. Client Reconnects to Same Server: If the client reconnects to the same compute server, the existing pub/sub connection is reused.
     • b. Client Connects to Different Server: If the client connects to a different compute server, the new server establishes a pub/sub connection and notifies the original server to transfer any pending events.
     • c. Session Timeout: A predefined session timeout occurs, after which the pub/sub connection is terminated.

5. Client Loop:

   • The client processes any received events and immediately initiates a new HTTP long polling request to continue receiving future events.

Sub tasks

• [ ] Design Event Interfaces and Structures
• [ ] Implement HTTP Long Polling
• [ ] Establish Inter-Server Communication
• [ ] Manage Connection Lifecycle
• [ ] Ensure Statelessness and Scalability

[thejhh]

For the moment, events can be implemented like any other end point and using 3rd party solutions, so not going to use time on this for now. Let's keep it on backlog.