[Question]: Why are meaningless words automatically filled in before the prompt?

yixuantt commented 10 months ago

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question

Here is my custom prompt.

qa_template = (
    "Below is an instruction that describes a task, paired with further context. "
    "Write a response that appropriately completes the request.\n\n"
    "Instruction:\n{query_str}\n\n Input:\n{context_str}\n\n Response:"
)

refine_template_str = (
    "Below is an instruction that describes a task, paired with further context. "
    "Write a response that appropriately completes the request.\n\n"
    "Instruction:\n{query_str}\n\n Input:\n{context_str}\n\n Response:"
)
refine_template = PromptTemplate(refine_template_str)
qa_template = PromptTemplate(qa_template)

And here is the generated prompt:

A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.\n### Human: Got any creative ideas for a 10 year old’s birthday?\n### Assistant: Of course! Here are some creative ideas fora 10-year-old's birthday party:\n1. Treasure Hunt: Organize a treasure hunt in your backyard or nearby park. Create clues and riddles for the kids to solve, leading them to hidden treasures and surprises.\n2. Science Party: Plan a science-themed party wherekids can engage in fun and interactive experiments. You can set up different stations with activities like making slime, erupting volcanoes, or creating simple chemical reactions.\n3. Outdoor Movie Night: Set up a backyard movie night with a projector and alarge screen or white sheet. Create a cozy seating area with blankets and pillows, and serve popcorn and snacks while the kids enjoy a favorite movie under the stars.\n4. DIY Crafts Party: Arrange a craft party where kids can unleash their creativity. Provide a variety of craft supplies like beads, paints, and fabrics, and let them create their own unique masterpieces to take home asparty favors.\n5. Sports Olympics: Host a mini Olympics event with various sports and games. Set up different stations for activities like sack races, relay races, basketball shooting, and obstacle courses. Give out medals or certificates to the participants.\n6. Cooking Party: Have a cooking-themed party where the kids can prepare their own mini pizzas, cupcakes, or cookies. Providetoppings, frosting, and decorating supplies, and let them get hands-on in the kitchen.\n7. Superhero Training Camp: Create a superhero-themed party where the kids can engage in fun training activities. Set up an obstacle course, have them design their own superhero capes or masks, and organize superhero-themed games and challenges.\n8. Outdoor Adventure: Plan an outdoor adventure party at a local park or nature reserve. Arrange activities like hiking, nature scavenger hunts, or a picnic with games. Encourage exploration and appreciation for the outdoors.\nRemember to tailor the activities to the birthday child's interests and preferences. Have a great celebration!\n### Human: ors.\nRemember to tailor the activities to the birthday child's interests and preferences. Have a great celebration!\n### Human: Below is an instruction that describes a task, paired with further context. Write a response that appropriately completes the request.\n\nInstruction:\nSummarize the text....

dosubot[bot] commented 10 months ago

🤖

Hello @yixuantt,

I'm here to assist you with your query. I'm an AI bot designed to help with bug solving, answering questions, and aiding contributors while waiting for a human maintainer.

From the code you've shared, it seems like you're using a custom prompt template for your LlamaIndex application. The "meaningless" words you're seeing before your custom prompt are likely part of the default prompt templates defined in the default_prompts.py file. These templates are used to structure the prompts that are fed into the model for generating responses.

For example, in the DEFAULT_SUMMARY_PROMPT_TMPL, the text "Write a summary of the following. Try to use only the information provided. Try to include as many key details as possible." is a part of the prompt template. This text is used to instruct the model on what it should do with the {context_str} that follows.

If you want to change these instructions or remove them, you can modify the corresponding prompt templates in the default_prompts.py file. Be aware that any changes to these templates will affect how the model generates responses. You can find the default_prompts.py file in the LlamaIndex codebase here.

I also found a similar issue titled [Bug]: Seeing Chain of Thought Prompts in Responses Using Azure which was last updated on July 11, 2023. The suggested solution is to switch to AzureChatOpenAI from gpt-3.5 when using Azure. Streaming is also supported in the LlamaIndex class.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

yixuantt commented 10 months ago

Happy to solve it.

dosubot[bot] commented 10 months ago

🤖

@logan-markewich, I could use your expertise on this one.

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

run-llama / llama_index