-
Love this repo, would love to contribute!
Feels like the most needed feature is to have a good evals on couple of test docs (Notion, Github..) to evaluate the incremental models trained.
I also …
-
````R
require(data.table)
require(mlr3mbo)
require(bbotk)
callback_plot = callback_optimization("plot",
on_optimizer_after_eval = function(callback, context) {
}
)
obfun = ObjectiveRFu…
-
### Describe the feature
[OpenAI/evals](https://github.com/openai/evals/tree/main/evals/registry/data) contains many community-sourced tasks, with about 400-500 datasets covering various languages an…
-
Node v0.12 got rid of `evals`, which the `kanso-utils` module was relying on. `kanso-utils` has been updated, but the `node_modules` folder is checked-into this repository, and is out-of-date.
Same …
-
In gitlab by @jamesmundia on Jan 27, 2017, 05:05
https://atmospherejs.com/mpowaga/autoform-summernote
-
This ADR will encompass all of the lessons learned and decisions made on how we will handle RAG-focused LLM evaluations.
These issues are encompassed in the [RAG Evaluations MVP Epic](https://githu…
-
If I have something like `@b rand(1000) sort!`, the first eval is much slower than subsequent evals within a given sample, which violates benchmarking assumptions and results in weird results. For exa…
-
### 💻 The Brief
AI Tech to use/discover:
- [TruLens-Eval](https://lablab.ai/tech/truera/trulens)
- [GPT-4 Turbo](https://clarifai.com/openai/chat-completion/models/gpt-4-turbo)
That is the cha…
-
Hi, cool project :)
I took a look at the evals and noticed that there's only 127 eval files. Further, only 107 of them seem to pass the tests.
Would it be possible for you to post the rest of th…
Naqu6 updated
1 month ago
-
Being the only thing even tangentially related to GPT-4 made open-source, OpenAI Evals can help assess the performance of an LLM on a growing set of tasks. Maybe this could be used to measure how clos…