GPT2 推理怎么使用beam_search，并同时输出多个结果呢？

bytedance / lightseq

LightSeq: A High Performance Library for Sequence Processing and Generation

Other

3.22k stars 329 forks source link

Open wotulong opened 2 years ago

wotulong commented 2 years ago

GPT2 推理怎么使用beam_search，并同时输出多个结果呢？有参考代码么？谢谢！

Taka152 commented 2 years ago

Lightseq doesn't support gpt2 with beam search currently, you can try topk sampling and topp sampling.