-
Hello!
I really liked your work on RLHF research. A very clear description in the technical report and a good implementation. I studied the entire code in detail and read the article several times.…
-
Being the only thing even tangentially related to GPT-4 made open-source, OpenAI Evals can help assess the performance of an LLM on a growing set of tasks. Maybe this could be used to measure how clos…
-
### SFT data
1. Started the SFT stage with publicly available instruction tuning data ([Chung et al., 2022](https://arxiv.org/pdf/2210.11416))
2. Fewer but high quality > Millions of data but low …
-
I find the reward function to be the most important part of RLHF, because it is the part which mimics a human evaluator, providing instant feedback to the model.
However, due to ChatGPT's wide rang…
-
> [rank0]: Traceback (most recent call last):
> [rank0]: File "/opt/tmp/nlp/wzh/LLM-Dojo/rlhf/rloo_train.py", line 167, in
> [rank0]: trainer.train()
> [rank0]: File "/home/nlp/miniconda3/…
-
chatGLM-6B is an open source model based on GLM fine tuned on over 1 trillion tokens of dialogue and RLHF for chat.
It's quickly becoming one of the most popular local models despite no good fast C…
-
I'm trying to train the actor model (BLOOM 1.5B) on a multi-GPU setup (3-V100s).
When I observe the GPU usage, only the GPU:0 is utilized and I run out of memory if I increase the batch_size.
Cou…
-
Hi authors,
First of all, thanks for your great work on LLaMA-2! This is an impressive work for open source large language models!
I have a question about section 3.1 in the paper, specifically …
-
No problems with Step 1 & 2, but for Step 3, it returns a runtime error: The size of tensor a (6144) must match the size of tensor b (8192) at non-singleton dimension 0
**PyTorch:1.12.1, 1.13.1, 2.…
-
### 🚀 The feature, motivation and pitch
Hi all, I was wondering if it's possible to do precise model device placement. For example, I would like to place the vLLM model on GPU 1 and let GPU 0 do othe…