Questions about the reranker model? Do i need to modify the xgboost reranker with different dataset?

[forestemperor]

as my title goes.

[zhiheng-huang]

1) In https://retriever.denser.ai/docs/experiments/mteb_retrieval, we stated that "For each dataset in MTEB, we trained an xgboost models on the training dataset and tested on the test dataset.". So yes, you need use different re-ranker models on different datasets to replicate the 15 datasets results reported in the url. 2) As MSMARCO dataset is a large dataset, you can try it first to see if it fits your use cases/data. 3) We may introduce a global model which will be trained on all datasets combined later.