-
Hi @alex-petrenko!
Thank you so much for the great project!
I am wondering whether there is a plan to include an evaluator component to the framework that is capable of periodically running the …
-
Do we come up with our own sort of bytecode/evaluation system, or maybe "just" use LLVM? My instinct would be to use LLVM. I think I'd opt for a simplified approach (especially, initially) to Google's…
-
## Describe the Bug
Puppetserver fails with `Function Load Error for function 'eyaml_lookup_key'` and produces error messages in the logs if puppet agent is upgraded to v8.7.0+. The latest version of…
-
Hey!
I have a freshly installed Windows 11 machine with Node 22 installed, when I try to run ``humanify local .js`` I get the following crash:
```
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: N…
-
### What version of CUE are you using (`cue version`)?
$ cue version
cue version v0.0.0-20240522083645-3ce48c0beadf
go version go1.22.3
-buildmode exe
-compiler gc
Defau…
-
Creating new evaluator configurations programatically fails unexpectedly. The reason is that in the default call, the correct_answer value is set to None instead to the default correct_answer.
```
r…
-
### Description
Very similar code (but different) to https://github.com/swiftlang/swift/issues/75711
but a different crash and backtrace
### Reproduction
```swift
import _Differentiation
stru…
-
When working with json data, then the camel-jsonpath language is useful to grab specific data or define how a json payload should be splitted with Split EIP etc.
I wonder if we can have a page in k…
-
Hi, thanks for sharing this benchmark.
One problem I met is when I evaluated the performance in math understanding using
`python3 -m axlearn.open_api.evaluator \
--input_file ./output/math_…
-
Hi! I am wondering if it's possible to use open source or self-deployed llms (and not only open ai) as the judge or evaluator? If yes, could you please refer to an example or part of the docs explaini…