Output results before non-linearity

[JonathanVacher]

Hi,

I'm doing computational neuroscience and image statistics. In these fields, some models give a probabilistic description for the outputs of filters before the non-linearity. It should be useful (at least for me) to have an option to choose the output we would like. In deepnet, it is possible to have the output at any level, however, the transforms have been learned and I would appreciate to use a standardized transform as the scattering transform is.

I tried to check the code myself but I can't be sure what to do without breaking the efficient implementation.

Thanks for your help,

[lostanlen]

Hello @JonathanVacher, This is not possible with the current Kymatio API, but is relatively easy to obtain by rolling up your own implementation of Scattering2D from scattering2d.py. What you need is for forward to return U_1_c and U_2_c from lines 205 and 219

https://github.com/kymatio/kymatio/blob/master/kymatio/scattering2d/scattering2d.py#L205

https://github.com/kymatio/kymatio/blob/master/kymatio/scattering2d/scattering2d.py#L219

Use a dictionary to store the arrays U_1_c for every n1, and U_2_c for every pair (n1,n2). The sizes of these arrays will be different.

Depending on the size of your image and your choice of J and L, this may exceed your available RAM, because there cannot be any subsampling in the absence of nonlinearities. This is why we did not make it part of the core API of Kymatio.

Tell us how it goes!

[JonathanVacher]

Thanks for your help !

I don't understand why there cannot be any subsampling. The subsampling is just a step and I'm not removing any steps.

[lostanlen]

@JonathanVacher you're removing the nonlinearity, which demodulates oscillations. Therefore, the output is no longer real and nonnegative, but complex-valued. Furthermore, because wavelets are analytical, they have zero average, and therefore the output of the scattering transform without nonlinearity will also have a zero average. If you were to subsample that output more than the scale of the wavelet at hand, you will end up with aliasing the result. This is why you can't sample all scattering path according to the the same rate. i hope this helps

[JonathanVacher]

Ok. To be sure I've understand well. The subsambling you're talking about is not part of the transform it's just an operation on the output, right ? It is the \phi_J of the kymatio paper ? Sorry for the confusion I thought this subsampling was part of the transform.

Will let you know how it goes.

[edouardoyallon]

Ok. To be sure I've understand well. The subsambling you're talking about is not part of the transform it's just an operation on the output, right ? It is the \phi_J of the kymatio paper ?

Yes, it is an operation on the output. Basically, you need to remove the calls to "modulus", "periodize", the for loop over 2nd order, the low-pass filtering and you will get a large array of an oversampled wavelet transform.

Feel free to close this issue if you feel that we answered to your question or to open a PR if you have an idea for an improvement!