watch_torch_model: only first training instance visible

deepkit / deepkit-python-sdk

12 stars 7 forks source link

watch_torch_model: only first training instance visible #3

Open stocyr opened 4 years ago

stocyr commented 4 years ago

watch_torch_model() does not work entirely. For all images, I somehow only see the very first training instance the model encountered, though the distributions and also some stuff in the weights still seem to get updated.

Setup:

Windows 10
Deepkit Release 2020.1.5
Deepkit SDK 1.0.1
Running experiment on anything, from local script run, to local UI start to deployed docker run, eg. on pytorch/pytorch:1.4-cuda10.1-cudnn7-runtime

marcj commented 4 years ago

Yes, this is on purpose. It detects the very first input used and uses this for further debug layer extraction. It wouldn't make much sense to use for every debug extraction a different input as you could not easily compare with previous debug state. However, you can change the input easily, see the documentation. https://deepkit.ai/documentation/python-sdk/model-debugger see "Custom input to visualize"