kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.47k stars 875 forks source link

Refactor the logging setup loggic #3896

Closed noklam closed 1 week ago

noklam commented 1 month ago

Description

I was pairing with @jitu5 and found that the logic of logging become quite complicated. @lrcouto @SajidAlamQB because you have made changes to this file recently. There is an opportunity to remove duplicate checking and overload variable default_logging_config.

Development notes

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

lrcouto commented 1 month ago

I like the changes. The messages about the logging itself are rightly informative.

astrojuanlu commented 1 month ago

logger.info will always emit logs for which logging file is read

👍🏼

noklam commented 1 month ago

Need a bit more thinking here: image

It's a nice idea but this is a chicken and egg problem. The log would not show up until we set the logging level. By default python show WARNINGs only so logger.info did log something but there is no easy way to actually see the log.

noklam commented 1 month ago

image Fix the annotation from `` to'` (for rich), delay the log after self.configure is run.