Open soyoung97 opened 8 months ago
Another example showing the weight of two models is exactly the same: (code from here)
I ran into the same problem, I tested the 10K and 100K monoT5 on BEIR and found the results were the same, and both were very low, close to zero
Hello, I was trying to evaluate the monot5-large model on the beir benchmark, but I noticed that the uploaded models - 10k and 100k models - has exactly the same model. (when I do diff on terminal, it shows no difference)![image](https://github.com/castorini/pygaggle/assets/29880214/3d7f9e8a-d5c6-4bee-880c-6f4b1de4b6f5)
Also, the performance of large models is a lot lower than the performance of base models - Could you check if the two uploaded large models are accurate? (maybe relevant to the previous issue)
Thank you!