search

project-baize / baize-chatbot

Let ChatGPT teach your own chatbot in hours with a single GPU!
https://arxiv.org/abs/2304.01196
GNU General Public License v3.0
3.15k stars 275 forks source link

Does prob_sum in greedy search always add up to 1? #18

Open qrdlgit opened 1 year ago

qrdlgit commented 1 year ago

https://github.com/project-baize/baize/blob/ccf0bb8485657b7c16a57456bbb835503bac2456/demo/app_modules/utils.py#L278

Was just looking through the code. Doesn't softmax ensure that all the elements add up to 1?

JetRunner commented 1 year ago

This is for top-p sampling. We mask out tokens outside top-p (by default 95%).