search

reactivemarkets / toolbox-cpp

The Reactive C++ Toolbox is an open source library of C++20 components designed for efficient, asynchronous network applications on the Linux platform.
https://reactivemarkets.github.io/toolbox-cpp/
Other
21 stars 20 forks source link

feat(log): support for asynchronous loggers #125

Closed markaylett closed 3 years ago

markaylett commented 3 years ago

Update logger components so that the underlying storage that backs the log stream buffer is dynamically allocated.

The dynamically allocated memory block can then be detached from the stream buffer and ownership transferred to the logger implementation.

The benefit of this approach is that it allows for efficient logger implementations that process the log message asynchronously on a background thread.

DEV-1325

rfernandes commented 3 years ago

Not sure I understand this change entirely, especially the bit "The benefit of this approach is that it allows for efficient logger implementations that process the log message asynchronously." This seems to remove some of the TLS storage (which went from static to allocated but isn't that just incurring a pointer lookup for the data now ?) Didn't the old implementation already have a logger thread anyway ?

markaylett commented 3 years ago

Not sure I understand this change entirely, especially the bit "The benefit of this approach is that it allows for efficient logger implementations that process the log message asynchronously." This seems to remove some of the TLS storage (which went from static to allocated but isn't that just incurring a pointer lookup for the data now ?) Didn't the old implementation already have a logger thread anyway ?

Toolbox does not provide an asynchronous logger; both std_logger and sys_logger synchronously write the log message to their respective backends (stdio and syslog).

The log message buffer was previously stored in TLS. The function signature for the logger backend was:

using Logger = void (*)(int, std::string_view);

The problem with this signature is that any asynchronous logger backend must make a copy of the log message. This PR changes the signature such that log message ownership is transferred to the logger backend:

using LogMsgPtr = unique_ptr<...>;
using Logger = void (*)(int, LogMsgPtr, std::size_t);

Asynchronous loggers are then free to transfer the message onto a queue for background processing without having to copy the message.

To make this work, however, the log stream implementation had to change so that log formatting operates on a dynamically allocated log message that can be released/detached from the log stream when the log line is flushed to the backend.

P.S. I should have liked to discuss this with you but I assumed you were still on your travels given the current situation.

rfernandes commented 3 years ago

Make sense now, thanks for explaining.