Open joonas-yoon opened 2 years ago
Hi @joonas-yoon π
This is an interesting feature idea! Here is some feedback:
What do you think?
Could you give me one example for multiple parameters? I have no idea about it but interesting.
and for second thing, RAM consumption, how about save all of its state and restore them?
obviously, it have to take more time and less performance. any idea?
Hey there π
Well, for multiple parameters, almost all layers π
from torch import nn
# Create a fully connected layer
layer = nn.Linear(4, 8)
# Don't track grad on the weights
layer.weight.requires_grad_(False)
# But the bias is still loose
for n, p in layer.named_parameters():
print(n, p.requires_grad)
which yields:
weight False
bias True
For the second part, I had the same in mind, I agree π
Oh I see. then, only for having different one, how about this?
_______________________________________________________________________________________________________________
Layer Type Output Shape Param # Trainable
===============================================================================================================
vgg VGG (-1, 1000) 0
...
| ββ3 Linear (-1, 4096) 16,781,312 False
| ββ4 ReLU (-1, 4096) 0 -
| ββ5 Dropout (-1, 4096) 0 -
| ββ6 Linear (-1, 1000) 4,097,000 weight: False
| bias: True
no matter it has multiple lines. it's okay with single line as like weight: False, bias: True
. but it prints too long string π€
Well, that will become hairy, I honestly don't want to spread on multiple lines. The only suggestion I can see is:
good, I totally agree with you.
one thing I want to suggest is, it needs to be noticed from documentation. for example, "False; contains partial mixed-trainable parameters"
π Feature
New column in summary,
Trainable
determines whether gradients need to be computed.We can know this from model's parameters easily:
In short, expected feature is:
Motivation & pitch
I have been trying transfering model with DenseNet, and got summary.
but there is no information which layer is trainable. this is the tail of result.
Alternatives
No response
Additional context
I will wait for your response. I want to hear what you think about this.