-
related to https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard but more meaningful scores
https://github.com/h2oai/h2ogpt/blob/ba6cad3207f8319b5c5f4b1e9099d7b909fdb661/generate.py#L132…
-
````R
require(data.table)
require(mlr3mbo)
require(bbotk)
callback_plot = callback_optimization("plot",
on_optimizer_after_eval = function(callback, context) {
}
)
obfun = ObjectiveRFu…
-
The problem:
```
research$main_cause_cat %>% unique()
[1] NA …
-
### Describe the bug
```
$ pip install evals
...
python3 -m venv .venvSuccessfully built fire langdetect
ERROR: Exception:
Traceback (most recent call last):
File "/home/dan/Build/evals/.…
-
--steps=100 train_epochs=1 epochs_between_evals=1
Can we change to "--steps=100 train_epochs=10 epochs_between_evals=1" in above line
-
`docs/custom-eval.md` discusses how to make an eval with custom code, however it is not mentioned anywhere in `docs/custom-eval.md` or `docs/build-eval.md` that these evals are not being accepted curr…
-
If I have something like `@b rand(1000) sort!`, the first eval is much slower than subsequent evals within a given sample, which violates benchmarking assumptions and results in weird results. For exa…
-
I wish I could find some rhyme or reason to this, but I can't. Here's a simple case...
If I eval (+ 1) with cpp, it almost always returns 1.
If I eval (+ "foo") it frequently just hangs. Though some…
-
### 🚀 The feature, motivation, and pitch
Collect together settings and commonly used reward models and evaluations. These RMs can be used for training time eval, but we would probably also want to …
-
Local evals take quite a long time, about 2 minutes on my rather fast machine. Often times however, you will have already eval'd some revision before. For example, running two reviews against master a…
Atemu updated
8 months ago