search

OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
https://optimalscale.github.io/LMFlow/
Apache License 2.0
8.11k stars 819 forks source link

RAFT Issues #868

Open jujipotle opened 3 days ago

jujipotle commented 3 days ago

Hello, I am trying to replicate RAFT, but am running into several issues when running ./scripts/run_raft_align.sh. I've tried both the main LMFlow branch and the LMFlow_RAFT_Dev subbranch, but both seem to be outdated. Have the authors or anyone else tried running RAFT and had it work? What branch was used to produce the results from the original RAFT paper?

research4pan commented 3 days ago

Thanks for your interest in LMFlow! The latest RAFT will be integrated soon (https://github.com/OptimalScale/LMFlow/pull/867). For temporary usage, you may try this script https://github.com/RLHFlow/Online-RLHF/blob/main/run_loop.sh by changing --loss_type sigmoid in line 33 to --loss_type raft first. Hope this information can be helpful 😄

jujipotle commented 2 days ago

Thanks for the quick response! So if I want to replicate the RAFT paper, I can only do so through RLHFlow, and not LMFlow?