-
If we are using local model, we need to pass device to utilise the gpu for inference. However, in `launch_concordia_challenge_evaluation.py `
```
# Language Model setup
model = utils.language_mode…
-
Thanks a lot for setting up this leaderboard! Some interesting results arrise already.
I was looking into specialized judge models, and I found theses ones:
- [ ] [Flow-Judge-v0.1](https://huggi…
-
Similar to #4582, but across different models.
This depends on #4580 for the evaluation.
-
I am trying to run this setup:
```
lm_eval --model vllm \
--model_args pretrained="Qwen/Qwen2.5-0.5B-Instruct",tensor_parallel_size=2,dtype=auto,gpu_memory_utilization=0.8 \
--tasks bbh_…
-
Hi, thanks for the great work! In evaluating the model, I see that the maximum number of objects is 1100 (please see code snippet pasted below). I am, therefore, wondering what happens when this limit…
-
- Outage probability
- Number of steps: print accuracy chart
- [score function](https://keras.io/getting-started/sequential-model-guide/)
-
Hi, thank you for your work. I noticed that the CLIP score reported in your paper is relatively low, around 17 (e.g., Table 1), while other papers (e.g., PixArt-alpha) commonly report scores around 27…
-
- **Is your feature request related to a problem? Please describe:**
The current face expression recommendation system uses MobileNet, and there is a need to evaluate a custom CNN model built from s…
-
Users are currently able to edit the following component-only fields: Manufacturer, Model or Version, Serial Number, and Hg Converter Indicator.
The changes are not saved to camdecmpswks.component af…
-
I am trying to reproduce the DAC training and evaluation.
I am getting negative values for SI-SDR loss even for the pretrained weights provided in the repo.
Could you please provide documentation f…