TorchEnsemble-Community / Ensemble-Pytorch

A unified ensemble framework for PyTorch to improve the performance and robustness of your deep learning model.
https://ensemble-pytorch.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.05k stars 95 forks source link

Validation loss is too high ..Am I calculating the loss properly? #147

Open singhabhinav opened 1 year ago

singhabhinav commented 1 year ago

Hi,

I am using VotingClassifier and logging out training and validation loss in TensorBoard to find out if my model is overfitting or underfitting. I had to modify the VotingClassifier code to calculate the validation loss and store in the tensorboard logs. Here is my logic which I am using for calculating validation loss. I have changed the logic in voting.py under VotingClassifier..please see line number 28.

Screenshot 2023-03-10 at 11 27 34 PM

Below are my TensorBoard charts..We can see that validation loss is too high even though validation accuracy is very good. I am using cross entropy as criteria and the number of estimators are 10. Can you please let me know if my logic of calculating validation loss is right in VotingClassifier

Screenshot 2023-03-05 at 8 40 00 PM