Closed lcw99 closed 1 year ago
Currently we do not have plans to implement beam search. On one hand, it would incur greater performance overhead, and on the other hand, if you want to generate natural text, beam search may not be the best choice.
Thanks for your answer. I really expecting Frequency and presence penalties functionality. Currently, I have some repetation problem on my model.
I am using this package, which shows much better performance than MS DeepSpeed-MII, which has a similar function. Do you have plans to implement beam search as well?