多卡并行无法model.generate

[louieworth]

我正在尝试多卡并行涉及到online generate生成新样本，但是似乎直接卡死，卡着不动。 如果我设置为单卡运行，就不会卡死，请问如何解决？

class Actor(nn.Module):
    def __init__():
    ....
    def generate(self, input_ids: torch.Tensor, **kwargs) -> Union[
        Tuple[torch.LongTensor, torch.LongTensor],
        Tuple[torch.LongTensor, torch.LongTensor, torch.BoolTensor],
    ]:

[hijkzzz]

please use vLLM generate.

[louieworth]

Any instructions how to use vLLM generate? I currently follow the class Actor(nn.Module) with generate function as my code follows DPO pipelines.