googleapis / python-logging

Apache License 2.0
120 stars 52 forks source link

[cloud.logging] CloudLoggingHandler does not set logName in Cloud Functions/Run environment #902

Open evil-shrike opened 1 month ago

evil-shrike commented 1 month ago

I'm setting up CloudLoggingHandler with a custom name to have an ability to filter log entries in Cloud Logging. But despite method I use to setup it up it doesn't work -

Given a Cloud Function (deployed as http triggered):

logger.py

import os
LOGGER_NAME = 'Sample'

logging.basicConfig(
    format='[%(asctime)s][%(name)s][%(levelname)s] %(message)s',
    level=logging.INFO,
    datefmt='%H:%M:%S'
)
loglevel = os.environ.get('LOG_LEVEL', 'DEBUG')
loglevel = logging.getLevelName(loglevel)

logger = logging.getLogger(LOGGER_NAME)
logger.setLevel(loglevel)

if bool(os.getenv('K_SERVICE')):
  import google.cloud.logging
  from google.cloud.logging.handlers import CloudLoggingHandler, setup_logging

  client = google.cloud.logging.Client()
  handler = CloudLoggingHandler(client, name=LOGGER_NAME)
  handler.setLevel(loglevel)
  logger.addHandler(handler)
  # also tried these:
  #setup_logging(handler)
  #client.setup_logging()

main.js

from logger import logger

def main(request):
  logger.info('Cloud Function is called')
  return ""

So I create a CloudLoggingHandler with my log name and added it to my Python logger. This is what suggested to do in official doc: https://cloud.google.com/python/docs/reference/logging/latest/handlers-cloud-logging But it doesn't work in Cloud Functions

Here's how my log entry look like:

image

The logname field is still default stdout.

evil-shrike commented 1 month ago

ok, this configuration seems to work:

client = google.cloud.logging.Client()
handler = CloudLoggingHandler(client, name=LOGGER_NAME)
handler.setLevel(loglevel)
setup_logging(handler)
logger = logging.getLogger(LOGGER_NAME)
logger.setLevel(loglevel)

but not all message from a Cloud Function are logged with the default background transport. As soon as I changed the transport to the SyncTransport I started seeing my messages:

from google.cloud.logging.handlers.transports import SyncTransport
handler = CloudLoggingHandler(client, name=LOGGER_NAME, transport=SyncTransport)

When it runs locally, it does flush the log queue:

Program shutting down, attempting to send 1 queued log entries to Cloud Logging...
Waiting up to 5 seconds.
Sent all pending logs.

but it seems not to happen while running in Cloud (Run).

The problem with initial code is that it doesn't use setup_logging (only logger.addHandler(handler)) and as so the default console handler is left and actually I saw output from it while messages from CloudLoggingHandler didn't appear at all.


Additionally, the reference documentation for CloudLoggingHandler says:

This handler is used when not in GAE or GKE environment.

which is very strange as definitely the handler supports GAE environment.

gkevinzheng commented 1 day ago

@evil-shrike What code are you running on the Cloud Function that gives you issues with not seeing all the logged messages?