Closed yuvalkirstain closed 3 years ago
Just wanted to add that I have the exact same issue and it is a critical one. This looks like a great framework, but the fact that I can't see how training is progressing in the form of a readable log (e.g. tensorboard, wandb, etc) makes it not useable. Even if some of the documentation could walk users through this (assuming it isn't just a bug) it would be helpful.
Apologies on the late response here!
Logging was added under conf/trainer/loggers
and can be enabled like such:
python train.py task=nlp/text_classification dataset=nlp/text_classification/emotion trainer.gpus=1 trainer.accelerator=dp log=true +trainer/logger=tensorboard
appending +trainer/logger=tensorboard
is the CLI command here. You can modify the save dir by appending trainer.logger.save_dir=my_directory/
. See the conf directory for more loggers! https://github.com/PyTorchLightning/lightning-transformers/tree/master/conf/trainer/logger
I'll make a PR to update the documentation, and give this information under a new tab to close this PR
Documentation has been added here: https://lightning-transformers.readthedocs.io/en/latest/trainer/logging.html
Great thanks!
🐛 Bug
The script crashes when I try to log the run.
To Reproduce
Run:
And then I get:
Expected behavior
Be able to run the script and log the training process!
Environment
conda
,pip
, source): pipAdditional context
Please help :)