Open ArshaanNazir opened 10 months ago
@ArshaanNazir could you share your updates?
@chakravarthik27 please share the latest updates.
Hi @JustHeroo,
We have finished benchmarking the paul_graham dataset and now I am working on creating curated datasets for retrieval evaluation to do the benchmarking embedding models. I plan to generate question-answer pairs for each dataset and implement metrics to evaluate embedding models.
@chakravarthik27 please update the latest status here.
@ArshaanNazir could you share your update
@Cabir40 Kalyan is working on it. He will be enhancing embedding benchmarks for other retrieval tasks. He is working on FinBERT-QA right now.
@chakravarthik27 is there any update?
Hi @Cabir40,
Still, it is in progress, I am currently working alone on the langtest project and implementing a high priority feature similar to Open LLM Leaderboard by Hugging Face(Eleuther). It may take some time to complete the embedding benchmarks.
@ArshaanNazir please add your self as an assignee.