bptripp
commented
5 years ago Here are some more papers on rotation/scale equivariance and invariance in CNNs:
https://ieeexplore.ieee.org/abstract/document/1699534/ https://ieeexplore.ieee.org/abstract/document/7899932/ http://openaccess.thecvf.com/content_cvpr_2017/papers/Worrall_Harmonic_Networks_Deep_CVPR_2017_paper.pdf https://arxiv.org/abs/1602.02660 https://www.sciencedirect.com/science/article/pii/S0925231218301644 https://arxiv.org/abs/1507.08754 https://arxiv.org/abs/1411.6369
The feature maps have dimensions of eccentricity and angle, rather than horizontal and vertical positions. So we do not have translation equivariance. Also, the image statistics may be different in the fovea and the periphery, due to different scales. These factors will work against the statistical efficiency of convolution. Other more subtle warping effects may cause problems as well. This should all be fixed.
Introducing rotation equivariance in the feature-map space might help to recover translation equivariance in image space. See https://arxiv.org/pdf/1711.07289.pdf re. rotation equivariace in CNNs via steerable filters.
Do deal with foveal-peripheral differences, it might help to use pairs of kernels that are averaged with different weights depending on eccentricity. It seems like this could go off the rails though depending on initialization. Maybe initialize the pair to the same values and let them diverge as necessary.