search

Guzpenha / transformer_rankers

A library to conduct ranking experiments with transformers.
https://guzpenha.github.io/transformer_rankers/
MIT License
161 stars 19 forks source link

Questions about hyperparameters for EACL2021 paper #3

Closed ddehun closed 3 years ago

ddehun commented 3 years ago

Hi. I have a question about the experimental setup on your EACL2021 paper, On the Calibration and Uncertainty of Neural Learning to Rank Models.

How did the number of forward pass for MCdropout (T) and the number of model for Ensemble (M) are set?

Thank you!

Guzpenha commented 3 years ago

Hi, we did not do any hyperparameter tuning for those and used 5 to be able to run experiments in a reasonable time. I believe previous work uses 5 as well (https://papers.nips.cc/paper/2017/file/9ef2ed4b7fd2c810847ffa5fa85bce38-Paper.pdf).