Raw prompting guide/explanation

[xenova]

Hi there! 👋 In reference to this,

The systemPrompt and initialPrompts options are not yet supported. Instead, you have to manually emulate these by using special control tokens in the middle of your prompt.

It would be useful to provide these tokens to the user as part of the model info. A bit of testing suggests that the tokenizer is similar to/the same as https://huggingface.co/google/gemma-2-27b-it, meaning we should be able to use tokens like <start_of_turn>, <end_of_turn>, <eos>, etc. However, I don't see how to prompt the model with these tokens correctly.

[niutech]

There are some examples in on_device_model_execution_config.pb:

User prompt: %s Output text:

Your task is to help a user write text to fill in a textbox on a webpage. You will be given some context about the page and a prompt from the user and will return the text to the user. User Prompt: Rewrite the following text to make it %x: %s Page url: %s Page title: %s Textbox Text:

where %x is: formal, less formal, shorter, longer

Your task is to help a user write text to fill in a textbox on a webpage. You will be given some context about the page and a prompt from the user and will return the text to the user. Page url: %s Page title: %s User Prompt: Rewrite the following text using different words but preserve the meaning, tone, and length: %s Textbox Text:

Your task is to help a user write text to fill in a textbox on a webpage e.g. a social media post, a review, or a form. You will be given some context about the page and a prompt from the user and will write down the post. Note that: 1) Output range should be roughly 3-5 complete sentences. If user specifies a specific length, respect that length (e.g., write a 2 sentence announcement about my new job). 2) The output should be in the same language as the user prompt. 3) If user is asking a question, do not answer the question and just elaborate on it. 4) Do not provide information about the user unless mentioned in the user prompt. Page url: %s Page title: %s Page contents: %s

[xenova]

Ah very useful! Thanks!

protoc --decode_raw < on_device_model_execution_config.pb

produces

1 {
  1: 6
  2 {
    1: "optimization_guide.proto.StringValue"
    2 {
      1: "%s <ctrl23>"
      5: 1
      6 {
        1 {
          2 {
            1 {
              1: 1
            }
          }
        }
      }
    }
  }
  3 {
    1: "optimization_guide.proto.StringValue"
    2 {
      1 {
        1: 1
      }
    }
  }
  5: 1
}
2 {
  1 {
    1: "Who is the first president of the US?<ctrl23>"
    2: "washington"
  }
  1 {
    1: "What is the first element in the periodic table?<ctrl23>"
    2: "hydrogen"
  }
}

suggesting <ctrl23> is the control token for end of turn? Although, I assume .prompt or .execute will still add these tokens afterwards, even if I specify them myself? Is that right @domenic?

[domenic]

This discussion appears to be about the Chrome implementation, and not the proposed API that this explainer is set up to discuss. As such, the best place for it is somewhere Chrome specific... https://crbug.com would be my best guess, but maybe @KenjiBaheux has a better place?

Let me close this so we can keep it clear that this explainer repository is focused on the proposal, not the Chrome implementation.