Open amir-abdi opened 1 month ago
A proof of concept work on the possibility of supporting the select grammar for OpenAI models. The mini experiment was conducted on GPT-4.
select
Idea: The idea is to extremely boost the logit_bias of tokens of all select choices to discourage any other token.
logit_bias
Flaw: It is not a fool-proof idea because a mixture of those tokens can still be selected to generate a new word which may not be among the choices.
A proof of concept work on the possibility of supporting the
selectgrammar for OpenAI models. The mini experiment was conducted on GPT-4.Idea: The idea is to extremely boost the
logit_biasof tokens of all select choices to discourage any other token.Flaw: It is not a fool-proof idea because a mixture of those tokens can still be selected to generate a new word which may not be among the choices.