-
I'm trying to run vanilla PPO against either a single reward model or an ensemble of 5 reward models.
Command: `accelerate launch --main_process_port=29503 --config_file configs/accelerate_config.…
-
https://github.com/InternLM/xtuner/blob/081c8ca874bdbf7a7f8cd0a9e4cba503eaaa7bba/xtuner/model/reward.py#L311
-
這個模型是從llama2 SFT出來的話,看llama2的論文似乎llama2並沒有經過RLHF(llama2-chat有),請問Taiwan llama2有經過RLHF的訓練嗎?如果沒有的話,有關繁體中文的對齊,可以使用RLHF來進行,而非SFT。至於comparison的資料集,可以考慮用ChatGPT來產生,這樣不知有沒有試過,謝謝
![image](https://github.com/…
-
Here is a tutorial for training a reward model: https://github.com/huggingface/trl/blob/main/examples/scripts/reward_modeling.py
If you try sweeping both `data_seed` and `seed` (both passed to `Rew…
-
### Prerequisites
- [X] I am running the latest code. Mention the version if possible as well.
- [X] I carefully followed the [README.md](https://github.com/ggerganov/llama.cpp/blob/master/README.md)…
-
### Describe the Question
Please provide a clear and concise description of what the question is.
我的 sh 如下:
CUDA_VISIBLE_DEVICES=1,3 python reward_modeling.py \
--model_type llama \
--m…
-
[paper](https://arxiv.org/abs/2305.18290)
## TL;DR
- **I read this because.. :** 배경지식 차
- **task :** RL
- **problem :** TRPO도 별도의 Reward model을 학습해야 하는데 모델이 커짐에 따라 너무 힘듦
- **idea :** rewa…
-
### Your current environment
```text
Versions of relevant libraries:
[pip3] flashinfer==0.0.9+cu121torch2.3
[pip3] numpy==1.26.4
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] sentence-transformers==3.0…
-
Hi,
first of all thank you very much for the repo!
I would like to ask, is the EOS token in the reward model dataset necessary for the model? I'm using a gpt2 model where EOS is used for BOS and …
-
I'm currently working on reproducing the training of NVIDIA's multi-objective architecture reward model. What are some questions about the training details of ARMO-RM? I'm using Mean Squared Error (MS…