Enable Powertools logging for imported libraries

[FredrikMeyer]

Is your feature request related to a problem? Please describe.

It would be nice to easily modify the logger of imported libraries to use JSON logging from Powertools as well.

Describe the solution you'd like

Something like discussed in this discussion: https://github.com/awslabs/aws-lambda-powertools-python/discussions/799

Maybe something like this?

logger = Logger(...., level="INFO", deny_list=["dont-log-this-library", "and-not-this"])

Describe alternatives you've considered See the linked discussion.

Thank you!

[heitorlessa]

hey @FredrikMeyer thanks for creating the feature request! I'm copying the high level items shared in the discussion on how to make this happen - We can tackle it after the upcoming release this week.

---

For this feature, it'd need to be slightly more involved. From the top of my head, it would need:

1) Add a new method in Logger to propagate its configuration to any loggers available, except the ones in the deny list 2) Add functional tests 3) Add docs explaining how to use it (we can do that)

Unsure on the UX yet, needs some iteration, but something like this:

from aws_lambda_powertools import Logger

logger = Logger()

logger.clone_config_to_all_loggers(level=…, deny_list=[“logger_named_X”, “botocore”])

This would copy the existing logger logging level & handler/formatter to all registered loggers except root logger (log duplication), and for those in the deny list.

Another area I'm unsure is when using log sampling feature - dynamically set log level to DEBUG on a % of function invocation - do we want to also propagate a DEBUG level to other loggers? I think the answer is no as it could bring too much noise vs actual application per se, as it's intrusive (implicit vs explicit).

[houbie]

looks good, but I would suggest a standalone function: def configure_standard_loggers(source_logger: Logger, level=None, includes=None, excludes=None)

from aws_lambda_powertools import Logger

logger = Logger()
configure_standard_loggers(logger) # clone for all loggers with same level as logger
configure_standard_loggers(logger, level=…, includes=[“logger_named_X”]) # only logger_named_X
configure_standard_loggers(level=…, excludes=[“botocore”]) # all except botocore

[heitorlessa]

Thanks @houbie that's what I'm leaning towards the most. One minor possible modification to make it less confusing is to support exclude only instead of both include and exclude. Naming is still the hardest :D.

Edit: Got an initial UX; will try the includes as I'm having second thoughts on how many loggers one might have registered due to dependencies one might be unaware of.

import boto3

from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging.utils import copy_config_to_registered_loggers

logger = Logger()
logger.info("test")

# copies handler and log level info from `logger` to `boto3`, `botocore`, `urllib`
copy_config_to_registered_loggers(source_logger=logger)

session = boto3.Session()
s3 = session.client("s3")

s3.list_buckets()

[heitorlessa]

Updating here to let you know that @mploski is taking this opportunity to get his first contribution - we'll let you know which release this will be part once he's ready... but you can also take a peek at the PR code for your own use

[FredrikMeyer]

Thanks for the update @heitorlessa 👍

[mploski]

@FredrikMeyer PR has been merged and this feature will be available in coming release. Here is the PR link if you would like to look at this: https://github.com/awslabs/aws-lambda-powertools-python/pull/927

[FredrikMeyer]

Thank you very much for the update. Appreciated!

[heitorlessa]

This is now released as part of 1.24.1 ;) https://github.com/awslabs/aws-lambda-powertools-python/releases/tag/v1.24.1

Lambda Layers will be published later today