search

OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
https://optimalscale.github.io/LMFlow/
Apache License 2.0
8.11k stars 819 forks source link

[Bug fix] Fix tokenizer multiprocessing in reward model #847

Closed wheresmyhair closed 3 weeks ago

wheresmyhair commented 3 weeks ago

Fix tokenizer multiprocessing in reward model (original issue: https://github.com/OptimalScale/LMFlow/issues/482) num_proc=4 1 num_proc=8 2