tomsib2001
commented
1 year ago I'm not sure that this is possible for chatgpt-3.5 (or any version of chatGPT), but for open LLMs, there is this technique called grammar-based sampling which basically forces the LLM to answer according to some predetermined grammar, here is a Hacker News thread and link to a ggml PR describing it: https://news.ycombinator.com/item?id=36819906 . This could at least force the LLM, when applicable, to only answer using a preset list of commands.
WIP
Current gpt-3.5-turbo performance is not very good due primarily due to bad understanding of function set (functions called at the wrong times, messages sent w/o call to
send_message, etc.). Goal is to upload specific variations of the key prompts (tailored towards worse instruction following ability of 3.5) that provide reasonable performance.