search

hpcaitech / EnergonAI

Large-scale model inference.
Apache License 2.0
630 stars 90 forks source link

num_beams for beam search #138

Open shammmmmless opened 2 years ago

shammmmmless commented 2 years ago

Hi, I want to use num_beams for generate, but PipelineModel can't. Can you support num_beams?

Best wishes.

dujiangsu commented 2 years ago

Since we are working on imporving a large-scale model demo recently, the requirement might be delayed for one or two weeks. You can use beam search in transformer and add it to your code following this. Also, welcome you to contribute your beam code usage to the project.