Implementing TG-Daemon Architecture

ammarfaizi2 commented 3 years ago

Good day everybody,

Today I have started to design GNUWeeb TG-Daemon architecture.

0. Short Intro

What is GNUWeeb TG-Daemon architecture? It's a daemon (background process worker) to handle an event from the Telegram webhook. Using this daemon we can process the incoming event from the background process and yield a response back to Telegram (as Telegram Bot).

1. Short technical overview of the TG-Daemon architecture

How does it work? In the front, we have a standby web server as an HTTP handler. This will be the first endpoint when the payload from the Telegram webhook arrives. After it arrives at the web server, the web server should build a packet based on the acceptor's protocol and send it to TG-Daemon.

Below the structure of the acceptor's protocol:

struct chan_pkt {
    uint16_t        len; // length in network endian format (__be16)
    char            data[8192]; // contains JSON payload from Telegram
};

After the acceptor receives the packet, it submits an SQE to be processed. Once the submission completes (enqueued to the queue), the connection between TG-Daemon and the web server can be closed immediately, and then the web server can respond to the Telegram webhook (whose sent the JSON payload) with 200 OK.

On the other side, we have an SQE dispatcher, it will consume the SQE from the head to tail and distribute it across multiple threads. Whenever a thread has finished its job, the CQE collector will collect the completion queue.

At the back kitchen, we have an event logger which records everything to the database. After CQE has been logged, we release the resource associated with it, so that it can be reused by the new incoming payload.

2. Visualization

Image can make everything more clear. tg-daemon-arch

3. Apologize

This isn't yet a working daemon yet, and anyone who hoped for that will have to wait for the first real release (1.0.0). Anyone with questions or suggestions is encouraged to chat or mail me.

Happy hacking!

Signed-off-by: Ammar Faizi ammarfaizi2@gmail.com

ammarfaizi2 commented 3 years ago

I am converting this PR to draft.

ammarfaizi2 commented 3 years ago

Ok, so this is ready for review. Does anyone want to review my pull request? I am ready to fight all your curses against my code. Because I am ready to answer all the technical criticism that you give.

I will merge it myself if no one cares, though.

ammarfaizi2 commented 3 years ago

FYI, I am not going to implement io_uring for this project in the near future because of stability issue I had with io_uring. Let's wait a bit longer until Linux 5.13 is released.

ammarfaizi2 commented 3 years ago

Ok, looks like there's no review needed for my code. So I am merging this into master branch.

GNUWeeb / GNUWeebBot