How to visualise features

[SumoYanyan]

Hi, Mila! I was confused by the feature visualization: 1.in MozafariDeep network, the conv3.weight's kernel size is small(about 5*5), but the visualized features are huge images. 2.I tried some feature visualization methods in CNN, but all methods require gradient (BP). I don't know if there is a magic method for STDP network's feature visualization.

3. The same confusions for KheradpishehDeep network. Thanks very very much~

[miladmozafari]

Hi Sumo, The visualization technique that we have used in our papers is a bit customized for our network and task. The idea behind this technique is to reconstruct features based on the weight matrices. For example, let the network has two convolutional layers (similar to the MozafariShallow). In the first layer, each feature map has a Gabor weight kernel. Now consider a weight matrix in the second conv layer. Each element in this matrix represents the synaptic weight connecting this neuron to one of the feature maps in the first layer (after passing through the pooling layer). Thus, a single element in this weight matrix can be extended into a larger Gabor kernel which is multiplied by the value of this element. We do this expansion for all of the elements in the weight matrix and combine them to reconstruct the feature. That's why the visualization is bigger than the weight kernel (we also perform some tricks to compensate pooling layers). In general, our method cannot visualize well if the network is large and has many feature maps. I recommend to write your own visualization methods with respect to your task and requirements. However, if you are interested in using the provided functions and our method, here is the guideline:

• get_deep_feature function in visualization module reconstructs features of a layer with respect to the (reconstructed) features of the previous layer. Therefor, if you want to reconstruct features of layer i with respect to features of layer j (i > j), you should call this function for all the layers from i to j.
• For each layer, you provide feature reconstructions of the previous layer as the first argument of this function. If you have stride in a layer, this stride affects the reconstruction and should be provided to the function.
• The third, fourth and fifth arguments are the info of the current layer for which you want to reconstruct the features. However, the last argument can be None if the current layer is a pooling layer.
• The function computes and returns the reconstructed features together with the accumulative stride. This way, you can easily use the output of this function as the first two arguments for reconstructing features of the next layer.

BTW, we used this function for our deep networks, however due to the high number of feature maps in each layer, we did not achieve good reconstructions. Again, I recommend to implement your own visualization which fits your needs and task.

[SumoYanyan]

thanks a lot!