-
### 🚀 The feature, motivation and pitch
I'm working on evaluating Llama3.1-70B on the [MMLU](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/mmlu/README.md) and [MMLU-Pro]…
-
: I have time and temperature as search parameters. When I use get_next_trial(), AxClient suggests both, which is fine for temper…
-
If I explicitly create a division by zero error:
```
[bvsm_dymola_fmi2:] Error: The following error was detected at time: 0
[bvsm_dymola_fmi2:]
Model error - division by zero: (nB*(k1*nA-k2*nC)) …
-
Not sure if this feature belongs to this library or would it require a complete separate library. I am proposing the creation of a library where llm benchmarks can be ran. For example, evaluating a mo…
-
Thank you very much for your work. I noticed that in the metric depth process, I did not evaluate.py script as I did before V1. I implemented the evaluation of kitti verification set according to the …
-
Hi,
Is is possible to provide a set of instructions to run evaluation on the validation set?
From the README:
> [test_eval.json](https://github.com/OpenDriveLab/DriveLM/blob/main/challenge…
-
When I use hpo as flaml, it can run normally according to the Quickstart code。But some errors occurred when I changed hpo to autosklearn. Is there something wrong with my code?
import pandas as pd
f…