-
**Context**
When running the evaluators over larger datasets, depending on the model, it is very common to run into LLM errors where the output is not valid JSON. For example, while running the ben…
-
Here is the result for [SemantiCodec](https://haoheliu.github.io/SemantiCodec/)
This is a 16Khz codec with three different bit rates:
1. For token rate 100 with book size 16384 the bit rate is 1.35 …
-
Hello!
Thank you for your work at MLLM.
I had a fine-tuning bug that I couldn't fix: when I ran the `stage2_sft.sh` script and trained with speech_conv_datasets only, the logger showed that the trai…
-
# Task Name
Vehicle sounds classification
## Task Objective
The primary goal of this task is to evaluate the audio language model's capability to accurately recognize and classify different …
-
Dear author, @LiheYoung , hello. The metric depth fine-tune really have baffled me these days:
I used my own datasets(sparse labels), and try to lower the lr of the pretrained model vitb or vitl…
-
# Task Name
Covid-19 Cough Audio Classification
## Task Objective
To develop and validate a machine learning model that uses audio cough recordings to accurately **identify and differentiate betw…
-
VLMEvalKit version: commit 8e0aace0504d952a25e310a1de66a32c2c1476f1
I added a custom MCQ format dataset to LMUData directory. It is successfully loaded and shows "UserWarning: Will assume unsupport…
-
Hello,
Recently, while working with this package, I encountered a problem.
I randomly split the test and training data into 20% and 80%. I also generated 10,000 separate pseudo-absence data points. …
-
Hello,
I have currently tweaked the prompt optimization tutorial so that I can see if I can improve it's ability to improve on medical multiple -choice datasets. However, the results are getting p…
-
Paper : [https://arxiv.org/pdf/2406.16860](https://arxiv.org/pdf/2406.16860)
Website : [https://cambrian-mllm.github.io](https://cambrian-mllm.github.io)
Code : [https://github.com/cambrian-mllm/cam…