Improve logging

[dpouris]

Issue: Improve Logging Mechanism

Description: The current logging mechanism is quite basic and lacks configurability and features. The manually invoked logs by the user, such as LogInfo, are not being adequately tracked or managed. To enhance the logging capabilities, a significant refactor is needed.

Goals:

1. Configurable Logging Levels:

   • Implement different logging levels (e.g., DEBUG, INFO, WARN, ERROR).
   • Allow users to set the desired logging level through configuration.

2. Log Formatting:

   • Provide options to format logs (e.g., JSON, plain text).
   • Include timestamps, log levels, and contextual information in log entries.

3. Output Destinations:

   • Enable logging to various destinations such as console, files, and external systems (e.g., log management services).
   • Allow configuration of log rotation and archival for file-based logging.

4. Contextual Logging:

   • Integrate request context (Ctx) into logs to provide better traceability and debugging.
   • Automatically log request metadata, such as request ID, IP address, and headers.

5. User-Defined Logs Tracking:

   • Track and manage manual logs (e.g., LogInfo) by the user.
   • Ensure these logs are subject to the same configurability and formatting as automatic logs.

6. Asynchronous Logging:

   • Implement asynchronous logging to avoid blocking request processing.
   • Ensure logs are flushed and persisted in case of application shutdown or crashes.

7. Error Handling in Logging:

   • Gracefully handle errors that occur during logging operations.
   • Provide fallback mechanisms to ensure critical logs are not lost.

Tasks:

1. Design Logging Interface:

   • Define a flexible and extensible logging interface.
   • Ensure the interface supports different logging levels, formats, and destinations.

2. Implement Configurable Logger:

   • Develop a logger that supports configuration for levels, formats, and output destinations.
   • Use dependency injection to allow easy replacement and extension of the logging mechanism.

3. Integrate Contextual Information:

   • Modify logging calls to include contextual information from the Ctx struct.
   • Ensure contextual logs are consistently formatted and tracked.

4. Refactor Existing Logs:

   • Refactor current logging statements to use the new logging interface.
   • Replace direct log writes with configurable logging calls.

5. Develop Tests:

   • Write comprehensive tests for the new logging mechanism.
   • Ensure tests cover different configurations, contexts, and error scenarios.

Impact: Refactoring the logging mechanism will significantly improve the observability and maintainability of the package. It will enable users to configure logging according to their needs, improve debugging capabilities, and ensure better traceability of user-defined and automatic logs.

Priority: High

Estimated Effort: Large

Dependencies:

• Configuration management
• Context handling in requests
• External log management services (if used)

Next Steps:

1. Design the logging interface and gather feedback.
2. Implement the configurable logger.
3. Integrate and refactor existing logging statements.
4. Develop and run tests to ensure robustness and reliability.
5. Document the new logging features and provide examples for users.

[kriptonian1]

I would like to work on this

[dpouris]

Hey there @kriptonian1! Of course, you can take a look at CONTRIBUTING and start from there. If you'd like we can collaborate together on this issue!

[kriptonian1]

SGTM