Seldaek / monolog

Sends your logs to files, sockets, inboxes, databases and various web services
https://seldaek.github.io/monolog/
MIT License
20.95k stars 1.9k forks source link

Trace Context for GoogleCloudLoggingFormatter #1877

Open iamacarpet opened 5 months ago

iamacarpet commented 5 months ago

Add Cloud Trace context to log entries formatted with GoogleCloudLoggingFormatter.

This allows log entries to be grouped with the HTTP request that generated them in GCP's Log Explorer:

Screenshot 2024-03-28 7 20 16 PM

@sl0wik @sonnysasaka - two big users of PHP on GCP, RFC please.

sonnysasaka commented 5 months ago

Not sure if we need this support directly at monolog level, but I've been able to use https://github.com/googleapis/google-cloud-php-logging, which is a PSR logger which is supported as a "Handler" by monolog, e.g. in this laravel snippet:

<?php
namespace App\Logging;

use Google\Cloud\Logging\LoggingClient;
use Monolog\Logger;
use Monolog\Handler\PsrHandler;

class CreateGcpLogger
{
    /**
     * Create a custom Monolog instance.
     *
     * @param  array<string>  $config
     * @return \Monolog\Logger
     */
    public function __invoke(array $config)
    {
        $loggingClient = new LoggingClient([
            'projectId' => env('GOOGLE_PROJECT_ID'),
        ]);

        $logger = $loggingClient->psrLogger('app', [
            'batchEnabled' => true,
        ]);

        return new Logger('gcp', [new PsrHandler($logger)]);
    }
}

And this already includes the trace id to each log entry: https://github.com/googleapis/google-cloud-php-core/blob/686ffd0bb1328cd2ee963c4444e2ab4b111ba3ba/src/Report/GAEMetadataProvider.php#L39

This works out of the box for app engine. For cloud run you might want to look at CloudRunMetadataProvider: https://github.com/googleapis/google-cloud-php-core/blob/686ffd0bb1328cd2ee963c4444e2ab4b111ba3ba/src/Report/CloudRunMetadataProvider.php

So this could be an alternative, if monolog prefers to stay agnostic towards formatting and let other modules define their own format to be used by monolog.

iamacarpet commented 5 months ago

Thanks @sonnysasaka ,

This came about from a conversation with @sl0wik who said his Firevel project was using this:

LOG_CHANNEL: stderr
LOG_STDERR_FORMATTER: Monolog\Formatter\GoogleCloudLoggingFormatter

in the environmental configuration for Laravel on App Engine Standard.

This method of logging more closely matches what is desirable in a containerised environment (logging via stdout/stderr) and works on PHP 7.4+ thanks to the PHP-FPM configuration option:

decorate_workers_output = no

Although I was concerned he & his users were missing out on logs being tagged together against the request that generated them, which our developers consistently say is their favourite developer friendly feature of GCP.

I believe that Google\Cloud\Logging\LoggingClient works by submitting to the Logs API endpoint directly, sped up by the Batch Daemon, which is only available on App Engine Flexible.

Using the stderr logging method along with this formatter allows for something that works across App Engine Standard, App Engine Flexible, Cloud Run, GKE and (potentially) GCE, without any latency associated with calling the Logs API directly.

Personally, we have our own log formatter as part of our GaeSupportLaravel package, however, adding the required stuff into here (since exists anyway) will hopefully bring this functionality to more people.

sonnysasaka commented 5 months ago

Thanks for providing more context. With that, I think it makes sense for the google cloud formatter to live in monolog repo itself. I don't have a particular concern with the code except one small suggestion inline in the patch.