Switch from `SGD` optimizer to `AdamW` for patch classification tasks

kaiko-ai / eva

Evaluation framework for oncology foundation models (FMs)

https://kaiko-ai.github.io/eva/

Apache License 2.0

66 stars 4 forks source link

Closed nkaenzig closed 1 month ago

nkaenzig commented 1 month ago

For some models we have observed SGD to not converge (fast enough), without further tuning the learning rate
This PR standardises the batch size for patch classifiction tasks to 256

roman807 commented 1 month ago

Thanks, looks good! Do you want to already include the updated leaderboard plots in this PR or would this come in a later one?