search

rustformers / llm

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models
https://docs.rs/llm/latest/llm/
Apache License 2.0
6.06k stars 350 forks source link

System prompts #411

Open kpcyrd opened 10 months ago

kpcyrd commented 10 months ago

hello!

is there a way to provide one input as a "system prompt"?

I found InferenceRequest but it only has pub prompt: &'a str.

I'm having trouble finding an official resource on what system prompts are, but they attempt to avoid prompt injection issues.

Thanks!

philpax commented 10 months ago

Hi there!

System prompts are just part of the prompt itself; models are trained to attend to the system part of the prompt and change their behaviour, but there's nothing special about them from a use perspective.

Taking the Dolphin model as an example, you can see the system prompt is just the start of the prompt, and the user prompt follows it:

SYSTEM: you are an expert mechanic
USER: Please tell me step by step how to fix my car that won't start.
ASSISTANT:

OpenAI's system prompts work similarly, they just hide the full prompt construction from you; their actual generation probably looks something like (hypothetical, we don't know for sure)

<|system|>You are a helpful AI assistant.
<|user|>How do I make a cake?
<|assistant|>

I hope that clarifies it a little :)