-
Currently, the evaluate_model function focuses primarily on accuracy and F1-score for classification models, and MSE and R² for regression models. We could enhance this by including additional evaluat…
-
@sanchit-gandhi
Where can I find faster-whisper model evaluation metrics? I don't see them on ASR leaderboard. Thanks!
-
I’ve also noticed that the evaluation of the regression model includes classification metrics such as accuracy, precision, recall, F1 score, and confusion matrix. These metrics are specifically desig…
-
I want to be able to perform post-evaluation query filtering after evaluating a model on a retrieval benchmark. In other words, after evaluation is ran I want to be able to select a subset of the test…
-
It would be nice if we could pre-compute a model's output on a particular dataset, and essentially "cache" this for use in an evaluation. For example, we have a large dataset of long-context documents…
-
I propose adding a Model Evaluation and Benchmarking System to ML Nexus to help users assess their model performance on standardized datasets and compare it against benchmarked scores. This feature wo…
-
```
ERROR:mteb.evaluation.MTEB:Error while evaluating BirdsnapZeroShot: 'VLM2VecWrapper' object has no attribute 'calculate_probs'
Traceback (most recent call last):
File "/data/niklas/mieb/m…
-
Hi, thank you for the great work.
Could you provide an evaluation script to reach the same metrics you reported on the paper, using the checkpoints model you kindly shared? Moreover, in another issue…
-
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
How to integrate Gemini models with RAGAS without faci…
-
Create and add some visualizations of the process to the hanging face and project presentation
Also can be presented with the tensorboard itself live during evaluation
Integrate tensorboard events i…