Handsome Log Transfer

[obenjiro]

Objective

The problem at hand is the complexity and difficulty in debugging HTTP requests made from a client to a Node.js Express server, especially when these requests pass through multiple middleware layers and backend APIs. This challenge is compounded by the asynchronous nature of Node.js, which makes tracing and logging these requests cumbersome and inefficient. Effective debugging is crucial for identifying and resolving issues, ensuring the reliability and performance of web applications.

Solution Proposal

We propose the creation of a tool named handsome-log-transfer that will be integrated with Express and injected into JavaScript. The primary function of this tool is to facilitate debugging of HTTP requests from the client to the Express server and through various middleware to backend APIs. By leveraging Node.js AsyncStorage, we can automatically augment every outgoing HTTP request, providing a functionality akin to a tracing tool.

Those traces will be logged to browser console.

In this example you can see that we can trace the reason why client HTTP request for /data get HTTP 500 error. First, we make request for /todos and then trying to go to wrong hostname *.com2 that doesn't exist.

Key Features:

1. Integration with Express: Seamless integration with Express to intercept HTTP requests and responses.
2. AsyncStorage for Context Management: Use Node.js AsyncStorage to maintain and propagate context across asynchronous calls, ensuring that logs and traces are accurate and complete.
3. Automatic Augmentation of HTTP Requests: Automatically add contextual information to outgoing HTTP requests, making it easier to trace the flow and identify issues.
4. Detailed Logging: Capture and log detailed information about each request, including headers, payload, response times, and any errors encountered.
5. Middleware Support: Support for various middleware layers to ensure comprehensive tracing through the entire request lifecycle.
6. Backend API Interaction: Trace requests from the server to different backend APIs, capturing the full request-response cycle.

Implementation Steps:

1. We need to discuss this tool. It already used in one internal project. Write me in direct so I can tell more about it (If we work together)
2. This tool is already published in internal repo. We need to publish it in github of gravity-ui
3. We need to integrate it to https://github.com/gravity-ui/expresskit

Technical Architecture:

Below is a Mermaid diagram illustrating the high-level architecture of handsome-log-transfer.

graph LR
    A[Client] -->|HTTP Request| B[Express Server]
    B -->|Middleware 1| C[Middleware Layer]
    C -->|Middleware 2| F[AsyncStorage Context]
    F -->|Patch HTTP/HTTPS Agents| G[Backend API 1]
    F -->|Patch HTTP/HTTPS Agents| H[Backend API 2]

    subgraph Express Server
        B
        C
        F
    end

    G -->|Response| F
    H -->|Response| F
    F -->|Response| C
    C -->|Response| B
    B -->|HTTP Response| A

PS: Tool with only work in DEBUG mode to avoid leakage of any private data (like tokens, private session variables)

[ykamendrovskiy]

@amje @korvin89 guys, what do you think?

[korvin89]

@obenjiro Hey! Sounds good. I suggest you to connect with maintainers of nodekit and expresskit

[obenjiro]

@resure What do you think about adding handlesome-log-transfer as built-in package for ExpressKit?

[resure]

I think that in its current state we should not include handsome log transfer in NodeKit or ExpressKit. However, it could be distributed as a separate library.

We have also met with Alexey and discussed potential ways to further develop this component, so we may reconsider this RFC in the future.