search

median-research-group / LibMTL

A PyTorch Library for Multi-Task Learning
MIT License
1.94k stars 181 forks source link

Question regarding the results in Table 1 of the paper #75

Closed IdanAchituve closed 5 months ago

IdanAchituve commented 5 months ago

Hi, Thanks for this great repository. Regarding the reported results in Table 1, did you take the best test values across training or the test values at the end of training? If it is the former, what metric did you use to chose the best model?

Thanks, Idan

Baijiong-Lin commented 5 months ago

We choose the best model on the validation dataset (or the test dataset if there is no validation dataset) in terms of $\Delta_p$ (a commonly used overall performance metric in multi-task learning, https://github.com/median-research-group/LibMTL/blob/main/LibMTL/utils.py#L48)

IdanAchituve commented 5 months ago

Great, thanks.