WuTi0525
commented
1 year ago @hamidehmoghadasi I have a different opinion on the question you raised.
- Regarding your question about the author's if statement in the test loop, I don't see anything wrong with it. Because the author divided the test into two cases: first, the neurons in the output layer can normally fire, which determines the classification by the time order of the spikes sent by the neurons in the output layer; Second, the neurons in the output layer remain silent during the time window, and then the size of the membrane voltage of the neurons in the output layer determines the category of classification. Both of these are reasonable, as long as we track the firing time of the neuron during the test phase, which is the data that must be calculated in the feedforward path of the neuron. These are reasonably fixed paradigm post-processing calculations, not bias or inference as you refer to them. And remove the outermost if statement, only rely on the spike timing to classify, the accuracy is not less than 92%, which is not the 10% you mentioned. Because classification by membrane potential is rare.However, this paper does not mention the accuracy of the film voltage test, which may be that the author wants to emphasize the advantage of classification through the output layer spike.
- The code does not add the training set to the test part, because the validation set is not necessary, and the source code uses the training set for training and the test set for testing. There is no such thing as to participate in the training of some images of the test set in advance to seek higher accuracy.
- I prefer what you say about supervised and unsupervised learning. Because the relative target time is really not the existing data, but the algorithm is calculated in the learning process, more like a way of unsupervised learning.
Here are 3 main problems in the code and paper which are basic and reject this paper and its results totally, unfortunately, the authors and the journal editors didn't take them into account.
The main problem is in the test loop, the authors imposed an if statement in order to handle cases where none of the output neurons fire, this condition directly impacts the value of the "correct" variable and biases the test accuracy based on a condition while introducing a condition in the test loop that biases the accuracy of the results is not a correct behaviour. The purpose of a test loop is to evaluate the performance of a program or algorithm by providing test samples and obtaining results without any interference or biasing. Biasing the results within the test loop made incorrect test accuracy (by removing the if statement from the test loop the test accuracy lowered to 10% which is too low) so all of the reported accuracies in S4NN are invalid.
The author uses the validation dataset as the test dataset and didn't spilt the training dataset into "Validation + Train" and then reported the test accuracy on MNIST datasets as 97.4% while the test dataset must be untouched and unseen by the model, by using a part of the training dataset as the validation, the test accuracy lowered to 93%.
The authors named the concept of "Relative Target" as the contribution of their paper in supervised learning, in which they move the target spike according to the output spike of neurons, this is against the concept of supervised learning where the teacher signal must be unchanged, teacher signal is a signal that the Error that we are going to backprop in the network is calculated based on it, here in this paper the Error signal is the time difference between the output single spike occurrence time and the target single spike occurrence time, so if they claim they proposed a supervised learning rule they must keep the target single spikes unchanged, now that they change the target spike time for each weight update they change the reference for calculating the Error value which is considered as an unsupervised learning rule, not supervised.