search

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.82k stars 895 forks source link

User research to see if we should remove the file-based logging handlers #1472

Closed antonymilne closed 2 years ago

antonymilne commented 2 years ago

Currently kedro logs to info.log and error.log files by default as well as to the console. Do some sort of user poll to find out how many people use the info.log and error.log files or whether we should just log to the console.

For context, see @noklam's comment here and following discussion: https://github.com/kedro-org/kedro/issues/1461#issuecomment-1105513363.

antonymilne commented 2 years ago

The results are in! image

image

And some comments:

I use info.log to log verbose information regarding the execution of an algorithm that I run in a node. If the output of the algorithm shows unexpected results, I can look in info.log to see what could have caused this unexpected result.

I use the logs in the directory mainly when running inside the docker container created with the kedro-docker plugin. When developing I mostly look at the console output even though it's useful to have a running history of previous executions even when developing

Conclusion

People do use the logs files. Let's leave them there.