-
Hi, I've found out that Pythia (from EleutherAI) is better that Cerebras GPT in terms of evaluation results. Pythia is basically a LLM that based on GPT NeoX architecture but it's parameters ranging f…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
the below is my code.
```
import torch
# from transformers import BitsAndBytesConfi…
-
## Sprint Planning - 2024-05-30 Week 6
### Checklist
- [x] #63 (to be done on Monday)
- [x] #91 (waiting for Tiffany's replies)
- [x] Prompt improvement by instructing GPT to distinguish stan…
-
### Current Behavior
from langchain.embeddings import HuggingFaceBgeEmbeddings
`model_name = "BAAI/bge-small-en"
model_kwargs = {'device': 'cpu'}
encode_kwargs = {'normalize_embeddings': False}
…
-
First off, I'd like to say thank you so much for publishing SWE-bench and SWE-agent. I was wonder is their anywhere that has the logs from running SWE-Bench/SWE-ENG evaluation are posted? I am working…
-
https://github.com/bazingagin/npc_gzip/blob/a46991564161023bba3b1267e0e74c69dab8f8eb/experiments.py#L116
It appears that in the `calc_acc` method it marks a sample correct if ANY of the labels with…
-
Hi, when I try to use holmes with a GPT-2 or Llama model, I get the following error:
`python investigate.py --model_name 'bbunzeck/gpt-wee-regular' --version holmes --parallel_probing --cuda_visibl…
-
**Submitting author:** @omlins (Samuel Omlin)
**Repository:** https://github.com/omlins/ImplicitGlobalGrid.jl
**Branch with paper.md** (empty if default branch):
**Version:**
**Editor:** @fcdimitr
…
-
-
### Feature request
Current trainer only supports teacher-forcing generation for computing evaluation loss but not auto-regressive generation for other metrics. Seq2SeqTrainer supports this but seems…