-
There's a lot of code in the product that performs a full refresh, usually by means of R evals, whenever the global R state is considered refreshed - in particular, at every new REPL prompt. When work…
-
The problem:
```
research$main_cause_cat %>% unique()
[1] NA …
-
### Describe the bug
When running evals such as the steganography eval, I get: AssertionError: Eval steganography.yaml not found
### To Reproduce
Install and set up everything according to th…
-
Compared to the rest of the runtime, the eval via nix-env takes the longest time when uncached.
It'd be nice if two checkouts could be made with two nix-env evals in parallel to speed this up.
T…
Atemu updated
2 years ago
-
Typing something in closh like `$(touch /tmp/test)` and pressing tab for completion will result in that code being executed (!).
It seems to affect completions for all shells: bash, zsh and even fi…
-
-
got the following on trying to use Dust. is there a dependency missing?
env: node 7.9
```
localDataPath: '/Users/dc/dev/rikai/tutorbot/papp/server/data' } +0ms
/Users/dc/dev/rikai/tutorbot/p…
dcsan updated
6 years ago
-
If we are running classification tasks with LLM, how can we calculate overall precision, recall and F1 score from the evals?
It is not clear if derived metrics allow us to do that. Any suggestions?…
-
When running
`python evals/lm_harness_eval.py --model mamba --model_args pretrained=state-spaces/mamba-130m --tasks lambada_openai,hellaswag,piqa,arc_easy,arc_challenge,winogrande --device cuda --…
-
In `BaseDelayedSaturatedMMM`, the `channel_contributions_forward_pass()` method calls `eval()` on the pytensor value to be returned ([here](https://github.com/pymc-labs/pymc-marketing/blob/3a9db517d1d…