search

2han9x1a0release / RBF-Softmax

MIT License
37 stars 7 forks source link

The loss does not decrease when I try to use RBF-SOFTMAX in my own code. #1

Open Hongbin98 opened 3 years ago

Hongbin98 commented 3 years ago

Hello, my question is: when I copy the "rbflogit.py" to my project and it is used to replace the "nn.Linear()", I get the right logits but the loss does not decrease. At the same time, I replace the "RBFLogits()" to "LinearLogits" and it works. So I wonder if there is any question in the optimization stage of the "RBFLogits()". Looking forward to your reply!

Hongbin98 commented 3 years ago

I am a newbie in this field, so please feel free to advise~

2han9x1a0release commented 3 years ago

Thank you for using RBF-softmax. Here are several suggestions:

  1. Trying to enlarge scale hyperparameter first; or
  2. Using more iterations/epochs for training, because some time RBF loss values changes a little in early stage; or
  3. Maybe you can give a brief description of your dataset as well as backbone. So that maybe I can give you more detailed suggestions.
392398741nm commented 3 years ago

Hello, I replace "nn.Linear" to "rbflogit.py" on MNIST. But the accuracy of the training set and the test set is always 0.1. Epoch was set to 80 and scale hyperparameter has been larged. If I use other loss functions, the net will work normally. I don't know why. Looking forward to your reply!