Question regarding the validation process to pick the best model

arthurdouillard / CVPR2021_PLOP

Official code of CVPR 2021's PLOP: Learning without Forgetting for Continual Semantic Segmentation

https://arxiv.org/abs/2011.11390

MIT License

145 stars 23 forks source link

Question regarding the validation process to pick the best model #36

Closed Ze-Yang closed 2 years ago

Ze-Yang commented 2 years ago

Hi, I find that the validation is done every epoch on the test set with only the current novel class annotations. However, it seems like the model will always be saved regardless of whether the validation mIoU gets better or not. That makes the validation totally in vain. Do you notice this problem? I think it was a legacy issue from MiB's code. Looking forward to your reply. Thanks.

arthurdouillard commented 2 years ago

I don't do early stopping.

You can use this option https://github.com/arthurdouillard/CVPR2021_PLOP/blob/main/argparser.py#L146 to have on validation and find the best hyperparameters.