-
### What is the idea?
@josephmure and I are trying to a build a Symbolic function with a sum with respect to the inputs.
To do so, we define explicitly a list containing all the inputs, but this see…
-
## Description
We need to choose a small number (1-3, depending on size) of open source RAG evaluation datasets. Having at least 1 open source dataset allows us to begin running basic evaluations tha…
-
When running
`python evals/lm_harness_eval.py --model mamba --model_args pretrained=state-spaces/mamba-130m --tasks lambada_openai,hellaswag,piqa,arc_easy,arc_challenge,winogrande --device cuda --…
-
The following is posted verbatim from @dtkerrs review of #49 with regard to the `switchy.distribute.MultiEval` interface and implementation:
> So some feedback here on `evals` is that it might be ni…
-
If you're using `eval` to capture errors, you have to at least report it. When posting a new post, the `eval` just swallows the error without reporting.
-
There's a lot of code in the product that performs a full refresh, usually by means of R evals, whenever the global R state is considered refreshed - in particular, at every new REPL prompt. When work…
-
In `BaseDelayedSaturatedMMM`, the `channel_contributions_forward_pass()` method calls `eval()` on the pytensor value to be returned ([here](https://github.com/pymc-labs/pymc-marketing/blob/3a9db517d1d…
-
**Is your feature request related to a problem? Please describe.**
I had to customize the client method "log_traces" to match my provided Spans. Therefore I have written a custom method, which I want…
-
-
Compared to the rest of the runtime, the eval via nix-env takes the longest time when uncached.
It'd be nice if two checkouts could be made with two nix-env evals in parallel to speed this up.
T…
Atemu updated
2 years ago