About Visual Encoder

[mountains-high]

Hi, thanks for the great work.

I wanted to ask a question about Visual Encoder.

In this Figure (taken from the paper), Could you please explain what do channels mean 45, 75 and 76?

I've taken the following sentences from Spiking Jelly documentations: Note that the size of the image input to the network is [1, 1, 28, 28],→the 0th dimension is batch, and the first dimension is channel therefore, when calling imshow, first use squeeze() to change the →size to [28, 28]

What is the difference between these two "channels"?

The second question is "Did you use Visual Encoder for both static and neuromorphic datasets(N-MNIST, DVS, etc.)?" Aren't neuromorphic datasets already in the spike train form?

Thank you very much for your time and considerations

[fangwei123456]

The input images are gray and only has one channels. But the output of spiking neurons layer has many channels.

Could you please explain what do channels mean 45, 75 and 76

I choose these channels because their output spikes are easily to read and understand, e.g., detecting the edges of cloth.

The images in figure 9 are chosen from the first spiking neurons layer's output spikes, whose shape is [N, C, H, W].

The codes in this tutorial (https://spikingjelly.readthedocs.io/zh_CN/latest/clock_driven_en/4_conv_fashion_mnist.html#visual-encoder) show all 128 channels:

out_spikes += encoder(img).squeeze()
            # the size of encoder(img) is ``[1, 128, 28, 28]``，the same use ``squeeze()`` transform size to ``[128, 28, 28]``

Aren't neuromorphic datasets already in the spike train form?

No, in fact, they are "sum of spikes". You can refer to this: https://spikingjelly.readthedocs.io/zh_CN/latest/clock_driven_en/13_neuromorphic_datasets.html#get-frames-data

[mountains-high]

Thank you for your detailed explanations.