-
**Is your feature request related to a problem? Please describe.**
When creating evals in Phoenix it would be convenient to be able to store / see a pass/fail marker for a specific eval. The `phoenix…
-
````R
require(data.table)
require(mlr3mbo)
require(bbotk)
callback_plot = callback_optimization("plot",
on_optimizer_after_eval = function(callback, context) {
}
)
obfun = ObjectiveRFu…
-
### Describe the feature or improvement you're requesting
Currently evals framework does not support Azure openAI implementation. This is blocker if someone wants to use eval with Azure OpenAI implem…
-
If I have something like `@b rand(1000) sort!`, the first eval is much slower than subsequent evals within a given sample, which violates benchmarking assumptions and results in weird results. For exa…
-
### Details
_No response_
Checklist
- [X] Modify `src/main.py` ✓ https://github.com/sweepai/evals/commit/79c50514c76cc63da87009fa58909bf838a262c9
- [X] Ran sandbox for `src/main.py`. ✗
- [X] Modi…
-
### Describe the bug
I used
> git clone git@github.com:openai/evals.git
and couldn't get `pip install -e .` to run.
Error logs said I was missing a setup.py or setup.cfg file. That's odd…
-
I have found it useful to have lifted infix operators to reduce the amount of parentheses and `Eval`s. For example:
```haskell
type List = Eval (Map SomeFunction '[SomeTypes]) ++ Eval (Map SomeFun…
-
PSA to users: Don't load untrusted input with Sorcar activated.
Sorcar uses the Python `eval` function a lot, and in unsafe ways. Pretty much everything involving arrays uses it, because "arrays" i…
-
There are use cases for mlrMBO beside computer experiments. In this case we have no objective function at hand. However, we might have an experimental design and we are interested in proposed points t…
-
Hello, it be great if we can pass in flag for disabling the opening explorer, I want to embed it in my club website where users just can see the moves and think on their own (learning purposes) than r…
jalpp updated
1 month ago