search

delve-team / delve

PyTorch model training and layer saturation monitor
https://delve-docs.readthedocs.io
MIT License
79 stars 13 forks source link

Plots from example.py are empty #49

Closed MLRichter closed 3 years ago

MLRichter commented 3 years ago

Fixed the bug in question. There are two factors with this. First, the example did not contain a validation step, which results in no validation result being plotted / the plots being empty, since not data is present. I also used this as an excuse to make the example more expressive, including things like metric logging and differentiated logging of validation and training saturation. Its only a few lines of code more but I think it helps to make thing clearer by quite a bit.

The second issue is located in Torch-Callback and concerns the case that a list of modules is handed to delve instead of a module containing submodules. If in this case, any of the modules is a recordable layer, delve will assign and empty string as a name to these layers, which will result in lost updated of covariance matrices during the forward pass and general inconsistent behavior. This was fixed by a small additional naming Scheme that ensures that no layers will not be assigned an empty string and that module names will be unique if contained in a list.

JustinShenk commented 3 years ago

Added CI to develop branch to confirm it passes.