-
https://github.com/EleutherAI/lm-evaluation-harness/blob/8138fd52437dcd8c76ac87bdc9d684840e794c42/lm_eval/tasks/leaderboard/ifeval/instructions.py#L1384
the updated IFEval dataset (https://www.oxen…
-
### Description
we need to train wav2vec2 model for specific speaker accent and compare the performance with the base model on test data of that particular speaker.
### Completion Criteria
A model th…
-
### This issue is for a: (mark with an `x`)
```
- [ ] bug report -> please search issues before submitting
- [X] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior …
-
## 🚀 Feature
Add new audio metrics for generative audio processing
### Motivation
The evaluation of speech processing (denoising, dereverberation and in general enhancement) highly depends o…
-
Hi team,
I have tried to use GPT4 as a classifier to classify the model responses but am getting content moderation filter trigger.
Changes made -- Instead of Llama70BJudge, I have a similar cla…
-
### Description & Motivation
`trainer.fit` only works with `CombinedLoader(..., mode="max_size_cycle"|"min_size")`
`trainer.{validate,test,predict}` only works with `CombinedLoader(..., mode="se…
-
**Describe the bug**
When attempting to create a new project by entering all the necessary details, an error message is displayed, indicating an issue with running the evaluation. The error seems to …
-
As a data scientist, I want to be guided in the choice of the arguments in the scikit-learn train_test_split function, without having too many warnings to avoid being over my cognitive budget charge (…
-
### Question
When I try to upload the vqav2 result file to the evaluation server https://eval.ai/web/challenges/challenge-page/830/my-submission, after I select the phase, the page will jump to the e…
-
Currently, the evaluate_model function focuses primarily on accuracy and F1-score for classification models, and MSE and R² for regression models. We could enhance this by including additional evaluat…