These models can't read your mind. If outputs are too long, ask for brief replies. If outputs are too simple, ask for expert-level writing. If you dislike the format, demonstrate the format you'd like to see. The less the model has to guess at what you want, the more likely you'll get it.
Tactics:
Include details in your query to get more relevant answers
Ask the model to adopt a persona
Use delimiters to clearly indicate distinct parts of the input
Specify the steps required to complete a task
Provide examples
Specify the desired length of the output
Provide reference text
Language models can confidently invent fake answers, especially when asked about esoteric topics or for citations and URLs. Providing reference text can help the model answer with fewer fabrications.
Tactics:
Instruct the model to answer using a reference text
Instruct the model to answer with citations from a reference text
Split complex tasks into simpler subtasks
Complex tasks tend to have higher error rates than simpler tasks. Decomposing a complex task into a workflow of simpler tasks can improve reliability.
Tactics:
Use intent classification to identify the most relevant instructions for a user query
For dialogue applications that require very long conversations, summarize or filter previous dialogue
Summarize long documents piecewise and construct a full summary recursively
Give the model time to "think"
Models make more reasoning errors when trying to answer right away, rather than taking time to work out an answer. Asking for a "chain of thought" can help the model reason its way toward correct answers more reliably.
Tactics:
Instruct the model to work out its own solution before rushing to a conclusion
Use inner monologue or a sequence of queries to hide the model's reasoning process
Ask the model if it missed anything on previous passes
Use external tools
Compensate for the weaknesses of the model by feeding it the outputs of other tools, such as text retrieval systems, code execution engines, or specific functions.
Tactics:
Use embeddings-based search to implement efficient knowledge retrieval
Use code execution to perform more accurate calculations or call external APIs
Give the model access to specific functions
Test changes systematically
Improving performance is easier if you can measure it. Define a comprehensive test suite to ensure that changes are net positive to performance.
Tactic:
Evaluate model outputs with reference to gold-standard answers
ShellLM
commented
2 weeks ago
Prompt Engineering - OpenAI API
Six strategies for getting better results
Write clear instructions
These models can't read your mind. If outputs are too long, ask for brief replies. If outputs are too simple, ask for expert-level writing. If you dislike the format, demonstrate the format you'd like to see. The less the model has to guess at what you want, the more likely you'll get it.
Tactics:
Provide reference text
Language models can confidently invent fake answers, especially when asked about esoteric topics or for citations and URLs. Providing reference text can help the model answer with fewer fabrications.
Tactics:
Split complex tasks into simpler subtasks
Complex tasks tend to have higher error rates than simpler tasks. Decomposing a complex task into a workflow of simpler tasks can improve reliability.
Tactics:
Give the model time to "think"
Models make more reasoning errors when trying to answer right away, rather than taking time to work out an answer. Asking for a "chain of thought" can help the model reason its way toward correct answers more reliably.
Tactics:
Use external tools
Compensate for the weaknesses of the model by feeding it the outputs of other tools, such as text retrieval systems, code execution engines, or specific functions.
Tactics:
Test changes systematically
Improving performance is easier if you can measure it. Define a comprehensive test suite to ensure that changes are net positive to performance.
Tactic:
Suggested labels
None