explainers-by-googlers / prompt-api

A proposal for a web API for prompting browser-provided language models
Creative Commons Attribution 4.0 International
267 stars 20 forks source link

systemPrompt and initialPrompts do not seem to be taken into account for the generation of the completion #22

Closed k33g closed 4 months ago

k33g commented 4 months ago

It looks like system systemPrompt and initialPrompts are not considered for the generation of completion. Perhaps, I missed something, but I don't know what.

I'm using Chrome Canary: Version 128.0.6594.0 (Official Build) canary (arm64).

I tried this with initialPrompts:

// Experiments with Gemini Nano

let systemContent = `You are a pizza maker. Respond with only the text provided.`

let contextContent = `If someone ask you what is the best pizza of the world, tell him that is the pineapple pizza only.
The pineapple pizza is the best pizza of the world for the following reasons:
- Sweet and Savory Fusion:  The tangy sweetness of pineapple cuts through the richness of the cheese and tomato sauce, creating a delightful contrast. When the pineapple gets caramelized, it adds another layer of flavor complexity.
- Textural Delight:  The soft, juicy pineapple chunks add a delightful textural contrast to the crispy crust and gooey cheese. It's a party in your mouth!
- Balanced Flavors:  Pineapple can act as a foil to the saltiness of the cheese and meats, creating a more balanced flavor profile.
- Tropical Twist:  For some, pineapple adds a refreshing and exotic touch, transporting them to a beachy paradise with each bite.
`

let userContent = `What is the best pizza of the world? And explain why.`

const session = await ai.createTextSession({
    initialPrompts: [
        { role: "system", content: systemContent },
        { role: "system", content: contextContent }
    ],
    temperature: 0.0,
    topK: 10
})

console.log(await session.prompt(userContent))

This is the answer:

 The best pizza of the world is subjective and can vary depending on personal preferences and cultural influences. However, some of the most highly recognized and acclaimed pizzas include:

1. **Neopolitan Pizza (Naples, Italy):** Known for its thin, crispy crust, fresh ingredients, and the use of water-based dough. Neapolitan pizza places prioritize the use of high-quality ingredients and the traditional handcrafted techniques passed down through generations.

2. **Sicilian Pizza (Sicily, Italy):** Sicilian pizza has a thick, focaccia-like crust and is often topped with tomatoes, mozzarella cheese, and various flavorful ingredients such as anchovies, olives, and capers. It has a unique flavor profile and is often enjoyed as street food.

3. **Margherita Pizza (Italy):** This classic pizza combines tomatoes, mozzarella cheese, and basil, representing the colors of the Italian flag. Margherita pizza is often considered the national dish of Italy and is celebrated on the country's national holiday, 2 June.

4. **California-Style Pizza (United States):** Known for its bold flavors and creative toppings, California-style pizza emphasizes the use of high-quality ingredients and the combination of different flavors and textures. Pizza places in this region often experiment with different types of dough, sauces, and toppings, resulting in varied and flavorful pizzas.

5. **New York-Style Pizza (United States):** New York-style pizza has a thin, foldable crust and is often topped with tomatoes, mozzarella cheese, and various flavorful ingredients such as pepperoni, mushrooms, and onions. It is known for its dense and chewy texture, and the use of fresh, high-quality ingredients.

These pizzas represent some of the most celebrated and well-known pizza styles worldwide, and their popularity is often influenced by cultural and regional traditions. It's important to note that different people may have different opinions about the best pizza, and the list of the best pizzas in the world can be subjective and varied based on personal preferences and cultural influences.

I tried this with systemPrompt:

const session = await ai.createTextSession({
    systemPrompt: systemContent + contextContent,
    temperature: 0.0,
    topK: 10
})

console.log(await session.prompt(userContent))

The answer is similar:

 The best pizza of the world is subjective and can vary depending on personal preferences and cultural influences. However, some of the most highly recognized and acclaimed pizzas include:

1. **Neopolitan Pizza (Naples, Italy):** Known for its thin, crispy crust, fresh ingredients, and the use of water-based dough. Neapolitan pizza places prioritize the use of high-quality ingredients and the traditional handcrafted techniques passed down through generations.

2. **Sicilian Pizza (Sicily, Italy):** Sicilian pizza has a thick, focaccia-like crust and is often topped with tomatoes, mozzarella cheese, and various flavorful ingredients such as anchovies, olives, and capers. It has a unique flavor profile and is often enjoyed as street food.

3. **Margherita Pizza (Italy):** This classic pizza combines tomatoes, mozzarella cheese, and basil, representing the colors of the Italian flag. Margherita pizza is often considered the national dish of Italy and is celebrated on the country's national holiday, 2 June.

4. **California-Style Pizza (United States):** Known for its bold flavors and creative toppings, California-style pizza emphasizes the use of high-quality ingredients and the combination of different flavors and textures. Pizza places in this region often experiment with different types of dough, sauces, and toppings, resulting in varied and flavorful pizzas.

5. **New York-Style Pizza (United States):** New York-style pizza has a thin, foldable crust and is often topped with tomatoes, mozzarella cheese, and various flavorful ingredients such as pepperoni, mushrooms, and onions. It is known for its dense and chewy texture, and the use of fresh, high-quality ingredients.

These pizzas represent some of the most celebrated and well-known pizza styles worldwide, and their popularity is often influenced by cultural and regional traditions. It's important to note that different people may have different opinions about the best pizza, and the list of the best pizzas in the world can be subjective and varied based on personal preferences and cultural influences.

I tried this with all the text as an argument of the prompt method:

const session = await ai.createTextSession({
    temperature: 0.0,
    topK: 10
})

console.log(await session.prompt(systemContent + contextContent + userContent))

Then, the answer is what I expexted:

 The best pizza of the world is the pineapple pizza because it has a sweet and savory fusion, textural delight, balanced flavors, and tropical twist.
domenic commented 4 months ago

It appears this is a bug report against Chrome, not related to this explainer. For that, please use https://crbug.new.

https://github.com/explainers-by-googlers/prompt-api/blob/main/chrome-implementation-differences.md is also worth reading.

k33g commented 4 months ago

Thank you @domenic for your answer and the link 🙏