OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

https://optimalscale.github.io/LMFlow/

Apache License 2.0

8.11k stars 819 forks source link

[Feature] Reward model inferencer support #866

Closed wheresmyhair closed 4 days ago

wheresmyhair commented 6 days ago

Description

Add reward model inferencer

Tests

This feature (rm inference)

Other features may be affected

Memory safe vllm inference

Finetune