Open lostanlen opened 3 months ago
Some questions came up after the review of #49 (which closed #48):
- should the theoretical bound be
T * (2**J)
or half of that? (i think it's half)
Sorry, I even didn’t notice it… Did you mean that the total theoretical receptive field should have a size of T * (2** J)
, but since only half of the input signal is plotted, the theoretical bound should be changed to T * (2** J)/2
?
- should the magnitude of the gradient be squared? (maybe not ...)
Probably I’m wrong: I think the concept of the effective receptive field is based on the forward/backward propagation process. If so, should we also consider the effects of the following operators? (For example, for a single layer, the activation function, and for the whole model, the loss function…?)
- should we aim for a
~ 1/T
power law? or~ 1/sqrt(T)
? or a constant? this has implications for the way we define theConv1D
operator in between DTCWT Direct and Inverse
Thanks! I think I further understand your intention by adding the power law curve!
Let's discuss and then make the appropriate edits to the script
Finally, maybe it’s not important: should we also operate a Conv1d
on x_phi
?
Probably I’m wrong: I think the concept of the effective receptive field is based on the forward/backward propagation process. If so, should we also consider the effects of the following operators? (For example, for a single layer, the activation function, and for the whole model, the loss function…?)
Oh... never mind this... I'll work on MuReNNDirect.to_conv1d
and also on its receptive fields' visualization!
Sorry, I even didn’t notice it… Did you mean that the total theoretical receptive field should have a size of T * (2* J), but since only half of the input signal is plotted, the theoretical bound should be changed to T (2** J)/2?
Yes, exactly what I meant. This is because we visualize the receptive field in logscale on the time axis.
Probably I’m wrong: I think the concept of the effective receptive field is based on the forward/backward propagation process. If so, should we also consider the effects of the following operators? (For example, for a single layer, the activation function, and for the whole model, the loss function…?)
I tried to keep things simple (linear and single-layer) in my script but generalizing to multiple layers is also interesting!
Finally, maybe it’s not important: should we also operate a Conv1d on x_phi?
yes, i would welcome that.
Now that we have MuReNNDirect.to_conv1d()
#56 it would be good to revisit these questions
Some questions came up after the review of #49 (which closed #48):
T * (2**J)
or half of that? (i think it's half)~ 1/T
power law? or~ 1/sqrt(T)
? or a constant? this has implications for the way we define theConv1D
operator in between DTCWT Direct and InverseLet's discuss and then make the appropriate edits to the script