-
As discussed during our @huggingface/datasets meeting, we are planning to move some "canonical" dataset scripts under their corresponding organization namespace (if this does not exist).
On the con…
-
Hello, I appreciate your kind words and efforts to replicate our results on my computer. Here is the process I followed:
First, I installed the necessary environment as per the "Installation" instru…
-
**What version of protobuf and what language are you using?**
Version: v3.8.0 (NOTE: please try updating to the latest version of protoc/runtime possible beforehand to attempt to resolve your pro…
-
see the thread by @lhoestq on Slack: https://huggingface.slack.com/archives/C034N0A7H09/p1652370311934619?thread_ts=1651846540.985739&cid=C034N0A7H09
```
pip install "bigbench @ https://storage.go…
-
Hi,
I have tried to run
```
accelerate launch -m lm_eval --model hf \
--model_args pretrained=${MODEL_PATH}/llama2_7b,max_length=2048 \
--tasks bigbench_generate_until \
--batch_si…
-
Hi, I am trying to evaluate my model on the BBH with/without CoT but all task results end up being 0.0. I am quite unexperienced, so please have it in mind when helping me out. Other tasks I've tried …
-
Hi,
Since Microsoft released [Orca paper](https://arxiv.org/pdf/2306.02707.pdf), we are likely to see models instruct-tuned using Orca techniques that'll most likely outperform the current crop of …
-
The goal of this segment is to create meaningful benchmark subsets with a minimal set of tasks.
I believe the steps are as follows:
1) construct an experimental subset. If people agree I can con…
-
We get:
```
Note: This error originates from the build backend, and is likely not a problem with poetry but with kenlm (0.2.0 https://github.com/kpu/kenlm/archive/master.zip) not supporting PEP 51…
-
track