-
Hi,
We are observing a significant performance decrease when we are running with CM scripts vs. running in command line of the model.
CM Run script on SDXL for 500 samples
```
cm run script …
-
The llava-next-video-34b DPO model is not performing well, whereas the 7B-dpo model works fine.
I've reviewed related issues and tried **_changing the conv mode to mistral_direct_**, but the respon…
-
With 1.6438261270523071 loss and 0.5646336674690247 accuracy, how could you get that predictions and confusion matrix so accurate.
-
Can you provide the evaluation code? When I tested it on the MMBenchmark with a 1B model, the performance was quite low, only around 19.
-
### Describe the bug
The performance for aggregation queries + SurrealML model seems very low, as compared to the scenario aggregation query + PyTorch/ONNX model. This behavior does not manifest when…
-
### Which API Provider are you using?
OpenRouter
### Which Model are you using?
Qwen2.5 Coder 32B Instruct
### What happened?
Cline currently lacks support for OpenRouter's custom routing feature…
-
Hello, and thank you for your excellent work! I am currently working on training a downstream task using LISA, but I’m unsure about the correct approach for training on the pretrained LISA model.
I…
-
Hello authors, thanks for your quick responses on my previous issues!
I'm making a new issue to ask whether these are the right hyperparameters for training the `bge-en-icl`. I'm finding that I ca…
-
Hi authors,
Thanks for sharing this amazing work! I'm wondering if you have any results on Davinci-002 model or any plan to evaluate this model, which is OpenAI's base model.
Thanks!
-
We should have benchmark folder where we add all benchmarking models that work on real-world datasets. For those models we want to track end-to-end performance and evaluation score. Also, we want to a…