Question about applying different log patterns to sinks within a single logger in Quill version 4

[hs-vc]

In Quill version 3, it was possible to apply different log patterns and time patterns to different handlers within the same logger. However, in Quill version 4, is it no longer possible to apply different log patterns to different sinks belonging to the same logger?

[odygrd]

Yes, that has changed in v4. The PatternFormatter has moved to the Logger.

Both designs have their pros and cons.

The codebase always had the restriction that there is a single Handler or Sink instance per open file as we are not using dup to have multiple Handler instances writing to the same log file, for simplicity reasons.

I have made this change in v4 because, for example, you might want different formats within the same log file. When the application initializes and logs versioning info, you might need a different format than during regular logging. With the previous design, you had to create a Handler with the initial format, then remove the Logger and close the Handler, followed by creating a new Handler with the same filename but a different format. All of this because of the above restriction one Handler instance per open log file. Moving the formatter to the Logger simplifies this process, allowing you to create a separate Logger class effectively writing a different format into the same file without having to interact with the open log file.

The drawback, as you noticed, is that you can't log once and have different formats as output. However, this limitation isn't too restrictive, as the Sink receives all the log Metadata along with the formatted message https://github.com/odygrd/quill/blob/master/quill/include/quill/sinks/Sink.h#L60

It is possible to ignore or manipulate the formatted message at the Sink level before output, effectively changing the format or creating your own format if a custom Sink is implemented. This is what the JsonFileSink is doing by not using log_message which is the formatted message coming from the PatternFormatter https://github.com/odygrd/quill/blob/master/quill/include/quill/sinks/JsonFileSink.h

If you let me know exactly what you're trying to achieve, I might be able to help with a workaround.

[hs-vc]

Thank you for the detailed explanation. What I aim to achieve is to create a logger with both a console_sink and a rotating_file_sink, each having a different time format and log pattern, as illustrated in the code below. This was straightforward in Quill version 3. Is there an easy way to accomplish this in Quill version 4? Any guidance would be greatly appreciated.

#include <iostream>
#include "quill/Backend.h"
#include "quill/Frontend.h"
#include "quill/LogMacros.h"
#include "quill/Logger.h"
#include "quill/sinks/RotatingFileSink.h"
#include "quill/sinks/ConsoleSink.h"

int main() {
    // Start the backend thread
    quill::BackendOptions backend_options;
    quill::Backend::start(backend_options);

    // Console sink
    auto console_sink = quill::Frontend::create_or_get_sink<quill::ConsoleSink>("sink_id_1");
    console_sink->set_log_level_filter(quill::LogLevel::Warning);
    std::string console_log_pattern = "%(time) [PID %(process_id)] [%(log_level)] [%(logger)] - %(message)";
    std::string console_time_format = "%Y-%m-%d %H:%M:%S.%Qms";

    // File sink
    auto rotating_file_sink = quill::Frontend::create_or_get_sink<quill::RotatingFileSink>(
        "rotating_file.log",
        []() {
            // See RotatingFileSinkConfig for more options
            quill::RotatingFileSinkConfig cfg;
            cfg.set_open_mode('a');
            cfg.set_max_backup_files(10);
            cfg.set_rotation_max_file_size(1024 * 1024);
            return cfg;
        }());
    rotating_file_sink->set_log_level_filter(quill::LogLevel::Info);
    std::string file_log_pattern = "%(log_level);%(time);%(logger);%(message)";
    std::string file_time_format = "%Y%m%dT%H:%M:%S.%Qus";

    quill::Logger *logger = quill::Frontend::create_or_get_logger(
        "root",
        {std::move(console_sink), std::move(rotating_file_sink)});

    logger->set_log_level(quill::LogLevel::Debug);

    LOG_INFO(logger, "This is a log info example {}", sizeof(std::string));
    LOG_WARNING(logger, "This is a log warning example {}", sizeof(std::string));
    LOG_ERROR(logger, "This is a log error example {}", sizeof(std::string));
    LOG_CRITICAL(logger, "This is a log critical example {}", sizeof(std::string));
    return 0;
}

}

[odygrd]

I have created an example here :

https://github.com/odygrd/quill/tree/single_logger_multiple_sinks/examples/single_logger_multiple_sink_formats

I plan to merge it into the master branch soon and include it in the next release. If there is anything you find unsatisfactory, please let me know, but I am confident it meets your requirements.

This example uses the Logger's pattern for the console sink and a custom format for the log file. You can easily reverse this setup, using the Logger pattern for the log file and implementing your own format for the ConsoleSink, if that suits your needs better.

[hs-vc]

Looks great! This example perfectly meets my needs. Thank you for your prompt help.

I have one question: I think it would be beneficial if the create_or_get_sink function could also accept a timezone parameter. Is this supported?

[odygrd]

great, thanks for confirming : )

create_or_get_sink accepts the constructor parameters of the sink you want to construct and forwards them to the constructor. In the example the default value for the Timezone from the constructor is used, but it will work if you provide your own

https://github.com/odygrd/quill/blob/d84224dcf6ae5f489b8c48261db6c6a8dd08cf2a/examples/single_logger_multiple_sink_formats/RotatingFileSinkWithFormatter.h#L15

[odygrd]

Since you are always logging Info to file and only Warning to console, it would make sense to reverse it.

https://github.com/odygrd/quill/blob/single_logger_multiple_sinks/examples/single_logger_multiple_sink_formats/single_logger_multiple_sink_formats_2.cpp

That way most log statements (info) will be formatted by default using the Logger pattern which is always applied.

For warnings only and above, the custom formatter in the ConsoleSinkWithFormatter will trigger the custom format.

That is slightly more efficient for the backend thread than the previous setup

[hs-vc]

Your suggestions make perfect sense. I appreciate your help. Thank you!