kddubey
commented
9 months ago I believe this should allow for selecting over long texts while removing the first tokens bias.
This is definitely an important thing to think about. Luckily, most language models are good enough that simply mentioning the options in the prompt can overcome the bias. So instead of doing—
prompt = "Make the right choice:"
options = [
" the first option",
" the second option",
" the final option",
]
gpt + prompt + select(options)—do:
options_as_str = "\n".join(options)
prompt_with_options = f"""Here are your options:
{options_as_str}
{prompt}"""
print(prompt_with_options)
# Here are your options:
# the first option
# the second option
# the final option
#
# Make the right choice:
gpt + prompt_with_options + select(options)Mentioning the options in the prompt causes the model to allocate a lot of probability to the first token of the correct option, and (after that token) it allocates a lot of probability to the second token of the correct option, and so on.
This prompting style costs more computation because there are more tokens in the context. But you'll almost always see an accuracy boost.
Another slight hack that works for heavily instruction-trained models: turn the task into a multiple choice question, i.e., point to each option (a long sentence) with a single letter. My notebook here demonstrates this strategy. Note that this prompt format may not work well if there are more than 5 options, because multiple choice question formats seen during training are usually limited to the letters from school exams: A, B, C, D, E.
i.e. does it work reliably on long options, for example choose between different sentences that may start the same way.
My notebook here evaluated guidance.select on a task where there are 77 multi-token options. It performs only slightly worse than other methods, and is significantly faster. The options in this task aren't full sentences. But hopefully the experiment gives you some confidence in the guidance.select algorithm.
At a high level, the algorithm should work well even if some or all sentences start with the same prefix.
⚠️ Take the info below w/ a big grain of salt. I haven't read the actual algorithm yet. This is just how I think about it.
Say there are 3 options which start with prefix, and we've reached that node in the tree:
prefix
/ | \
s1 s2 s3s{i} is the first token of the suffix of the ith option. From here, pick the suffix child token (s1, s2, or s3) with the highest conditional probability/logit (given the prompt + prefix). If this suffix child token has no children (or all of its children have at most 1 child), we're done; the option is unambiguous. Else, recurse on that child.
All this to say that the algorithm distinguishes between options with the same prefix by picking the option whose first suffix token is more likely. This heuristic is pretty in line with how greedy decoding in text generation works, so it should work pretty well.
I want to select solely on p_n
Interesting idea. I'm not sure this works well though, as p_n might be really high simply because token_n is really probable given tokens 1, 2, . . ., n-1 (regardless of the prompt!). In case you'd like to explore a more holistic method, check out (my project) CAPPr. It takes the average log-probability of tokens 1 through n. It also has functionality to let you explore the argmax p_n idea. Feel free to open an issue there if you need help :-)
CorentinvdBdO
CarloNicolini
Hey thanks for your work, guidance is amazing to work with now!
I had a question regarding select. As I understand it is simply a bias of logprobs over the tokens. Also, LLMs don't predict blindly token by token but instead build a "tree" of n tokens and use the most likely one. So does select work in the same manner? i.e. does it work reliably on long options, for example choose between different sentences that may start the same way.
To be clearer, if we represent an option as a list of tokens: option A: [token_1, .... token_n] The logprob associated: p(token_1), p(token_1 & token_2), .....p(token_1 & ...... & token_n) written as p_1, .... p_n
I want to select solely on p_n , and not select on p_1, then look for token2, etc... I believe this should allow for selecting over long texts while removing the first tokens bias.
Please tell me if I misunderstood something. Thanks again for your key contribution to open research!