[feature request] add nbest option to top-k / top-p (nucleus sampling) generation

OpenNMT / OpenNMT-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

https://opennmt.net/

MIT License

6.76k stars 2.25k forks source link

[feature request] add nbest option to top-k / top-p (nucleus sampling) generation #2313

Open vince62s opened 1 year ago

vince62s commented 1 year ago

This option would enable to re-rank nbest hypotheses generated with nucleus sampling.

PC91 commented 1 year ago

Hello @vince62s, I would like to know if this feature is already implemented. I am also interested and I could give it a try. Thank you.

vince62s commented 1 year ago

It is not implemented yet. I recently revamped beam_search.py and you can check where we use n_best to make sure we add enough hypotheses. You can easily do the same in greed_search.py it is not difficult, if you have issues let me know.