Closed JewelChen2019 closed 2 years ago
Hi,
Our final teacher is an ensemble of 3 models: gpt2-xl, gpt2-xl-F (finetuned) and w2v. You can check the performance of individual models in Fig. 2 from our paper. The performance of the ensemble is depicted in Fig. 6a from our paper. Overall the performance of this teacher is below the performance of TeachText.
Cheers!
I wonder if the teacher models perform better than the student model when doing retrieval?