-
-
When runnnig `venv/bin/benchmark-run -r boolq:data_augmentation=canonical,model=ai21/j1-grande -m 50` I get a `logprob` of 0.0 for all instances (which can't be right since it means probability of 1.0…
-
cohere, huggingface, ai21
-
```
Traceback (most recent call last):
File "/u/nlp/anaconda/main/anaconda3/envs/crfm_benchmarking/bin/helm-summarize", line 8, in
sys.exit(main())
File "/juice2/scr2/nlp/crfm/benchmarkin…
-
-
## Description of the feature request
Related to #29, it would be very nice if this were a lightweight package that were available to `pip install`.
## Description of the solution you'd like
…
-
New pricing model: https://studio.ai21.com/pricing
-
Been seeing a lot of YouTubers adding LangChain agents for giving babyagi more functionality for its task completion. This issue is similar to #96, but I wanted to go a bit further with adding Langcha…
-
Hey, using new models such as EleutherAI's and NovelAI's [Genji](https://huggingface.co/NovelAI/genji-python-6B) or apis such as AI21 or OpenAI, a completer could actually sometimes complete an entire…
xloem updated
2 years ago
-
Tokenizers have been renamed in https://github.com/stanford-crfm/benchmarking/pull/633/files#diff-d51d49420311f844f65c0edf5760f86d4f17a862c5601c223bc0dcce14be3940, from "huggingface/gpt2_tokenizer_fas…