Closed borisdayma closed 2 years ago
Hi!
We recently redesigned the internal logging logic. Was this something supported before? Can you provide a minimal repro to play around and test it?
self.log
is designed to support numbers, tensors, or dictionaries of the former. To support this we would either need to provide another function for this (e.g. self.log_data
) or add a flag to self.log
so most of the logging logic is skipped (e.g reductions and aggregations).
Hi, yes there is a colab in the issue under "To Reproduce" section.
Hey @carmocca , @borisdayma,
I don't believe we ever supported logging images directly with the self.log function.
I think having self.log_image
or derivate would be the cleanest way forward.
Best, T.C
It works with latest released version on the exact same notebook: run
Hey @borisdayma,
Interesting, I don't understand how this could have worked :) Need to investigate.
Best, T.C
@borisdayma as a workaround, in the callback could you directly log with trainer.logger.experiment.<wandb_api>
?
Yes this would work to access directly the correct logger.
The main issue I see is that several people use the self.log()
pattern which technically is just a shortcut for self.logger.experiment.log()
with WandbLogger
.
It has also other advantages as it works when there is no logger attached to the trainer yet (self.log()
just becomes a noop) for example when using the learning rate finder prior to training.
Here is a cool notebook which was working a short time ago on master and that I planned to showcase for next release of pytorch-lightning master (I guess v1.4).
Just to let you know, for now I replaced the pl_module.log(examples, [wandb.Image(…)])
with wandb.log({'examples': [wandb.Image(…)})
in my colab.
Hi guys, is there a way to log a Confusion Matrix using the build-in .log
function? Since you have to pass a reduce function and you can't specify the axis the resulting logged metric will always be a scalar.
Hey @borisdayma,
Thanks, I bumped this one to P0 as it was definitely a great feature. We need to investigate how to properly support this again.
Best, T.C
Cool, let me know how I can help!
Hey @borisdayma,
After brainstorming with the Lightning Team, we decided to close this issue for the following reasons:
This change has already in place for 2 major releases, so the log API supporting tensor, metrics is considered as a stable API now.
A possible idea would be for the LightningModule to support log_{}
, but we don't believe there is a way to support all the underlying data API from the loggers in a consistent way + would add maintenance cost on the Lightning Team.
Best, T.C
Thanks for the feedback. Ping me back if you want to support it at some point.
🐛 Bug
Function
__check_allowed
limits the type of parameters that can be logged, even this data would be supported by loggers such as wandb.To Reproduce
See following notebook.
Error stack:
Expected behavior
This data is supported by wandb logger which can log it (and display it) with no problem (and is used in such way by many users).
The validation data type should be associated only to loggers are some are more flexible than others.
Environment
Additional context
More and more data types are supported by wandb (3d graphs, audio, custom data types, etc) so it would be good to be able to take advantage of it.
A workaround would be to log without using
LightningModule
(and using directlywandb.log
) but we lose some interesting lightning features (such as auto-tracking ofglobal_step
during logging) and also lose backward compatibility with existing code.