-
Should be quite easy to add for someone who knows the codebase. The biggest problem might be a new dataset format.
Don't expect I need to link this but it's pretty nice implementation of the loss:
…
-
@Ablustrund 我之前使用其它框架训过RLHF,发现模型可能在2000 step的时候效果相比sft是有所提升的,但是继续训练,可能到第3000个step的时候就完全训崩了,请问你们在做实验的时候有遇到类似的问题么,就是增加训练的step,后续容易训崩,这个可能的原因是什么呢
-
![98DDB13F-60AE-4F7D-8979-9B287A2A4CC1](https://user-images.githubusercontent.com/39515647/233412075-f68a9c2b-24c8-426c-80d3-6f2c0e48b1ca.png)
-
Hello, I am a student in Korea working on a 6 week project.
I want to fine-tune a CodeLlama model using your paper's methodology for the Code Repair task. How do you estimate the GPU resources and …
-
# URL
- https://arxiv.org/abs/2310.05914
# Affiliations
- Neel Jain, N/A
- Ping-yeh Chiang, N/A
- Yuxin Wen, N/A
- John Kirchenbauer, N/A
- Hong-Min Chu, N/A
- Gowthami Somepalli, N/A
- …
-
### Your current environment
We are working on accelerating RLHF algorithms and need to broadcast the weights of the DeepSpeed engine to the vLLM Ray worker. In v0.4.2, we were able to create an ad…
-
OpenAI used **40 people** when training their own chatGPT, and the annotation process lasted for **3 months**.
It is difficult for our open source community (github) to reproduce the **Reinforcemen…
-
**Is your feature request related to a problem? Please describe.**
We have posted a paper with codes [RRHF] (https://github.com/GanjinZero/RRHF) that can achieve human alignment without RLHF. RRHF ne…
-
### 🐛 Describe the bug
I am following this blog https://medium.com/pytorch/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b to train a 6.7B paramete…
-
https://arxiv.org/abs/2203.02155