search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
135.32k stars 27.08k forks source link

Saving logs to file #34610

Open techkang opened 2 weeks ago

techkang commented 2 weeks ago

Feature request

Add a FileHandler to root logger.

Motivation

Transformers is only using StreamHandler for logging, which makes debug analysis hard across different experiments. Many other frameworks support both logging to terminal and to file. For example: fairseq: https://github.com/facebookresearch/fairseq/blob/ecbf110e1eb43861214b05fa001eff584954f65a/fairseq_cli/train.py#L63-L65 detectron2: https://github.com/facebookresearch/detectron2/blob/8d85329aed8506ea3672e3e208971345973ea761/detectron2/utils/logger.py#L108

Your contribution

I can propose a PR.

LysandreJik commented 4 days ago

If you feel like offering a PR, that would be great! It can likely just be a doc PR showing how to add a new logging handler serializing the logs to a file.