search

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.91k stars 900 forks source link

Write documentation on how to make a custom logging.yml in your project #1474

Closed antonymilne closed 2 years ago

antonymilne commented 2 years ago

Once logging.yml has been removed, we need to add to the docs how a user can create a custom logging.yml in their project conf. This should include one or more nice copy and pasteable examples that demonstrate things a user might actually want to change in their logging (e.g. change the logging level maybe?).

Note the incremental option discussed on https://docs.python.org/3/library/logging.config.html. Would be nice to show how that can be used to simply edit the logging config rather than needing to overwrite the whole file. See https://github.com/kedro-org/kedro/pull/1491/files#diff-3434028fb098b270c88bc7a0689c0e4bc358442e8f29672ddb6bf87c11a0e28f for an example of how this works. Ask me to explain...

Makes sense to be done after #1469.

datajoely commented 2 years ago

So much like the project CLI being removed from the template I'm fine with this living docs.

However what do we think about a kedro template {thing} command?

We could use it to create templates for custom CLI, logging, datasets etc. a bit like we do for kedro pipeline create.

Thoughts?

noklam commented 2 years ago

I think it could help but I feel it is something that falls into the "nice to have" category. Although looking at the statistics seems people do use kedro pipeline create? Maybe I am wrong here.

I just found that the cli.py is also outdated, maybe another sign that not many people are actually using it.

datajoely commented 2 years ago

kedro pipeline create is the best way to create pipelines, the idea above is very much a 'nice to have'.

cli.py is a funny one as I think we've made it quite hard to change, I'm not sure it's super well documented how you create your own. Like I would expect to see it here, but it's not there.

noklam commented 2 years ago

It's actually here. Actually, I am only able to search it with the keywords cli.py, I suspect a normal user will struggle to find this information.

I am not sure how many projects will need the cli.py for a project base usage, plugins are probably the way to go for cross-projects.