Copilot functionality

[Ed-cred]

Hey, I've been using this project for a while an creating different agents for random use cases. From the description of the copilot mode visiting the websites it finds as relevant results of splitting up a query into multiple rephrased variants and navigating those sites seems like a very good tool for searching through restaurant menus that usually come in many different forms.

From what I could gather this functionality is not currently implemented in the project, but I would like to assist in its development as it is a tool that would be very good for my use case. Is there any way I can contribute to this, or if there has not been any work done on this already could you provide any guidance as to where to start with the implementation?

[Zirgite]

Enhancing Search Capabilities with AI Copilot functionality.

Main idea: Perplexica Copilot AI-assisted search functionality designed to understand and refine user queries to generate more accurate and comprehensive search results. The Copilot leverages a two-phase approach—self-define and self-refine—to iteratively improve the quality of search outcomes.

Key Phases:

Self-Define Phase:
    Objective: To break down the user's query into actionable components and plan multiple searches to cover all aspects of the query.
    Process:
        Understand the Query: Analyze the query to identify its key components and the specific information required.
        Formulate Search Plans: Create one or more tailored search plans that address different facets of the query.
        Execute Initial Searches: Conduct the initial searches based on these plans, ensuring a broad coverage of the topic.

Self-Refine Phase:
    Objective: To evaluate the initial search results, identify gaps or inaccuracies, and refine the search strategy.
    Process:
        Evaluate Initial Results: Critically analyze the search results for relevance, accuracy, and completeness.
        Identify Areas for Improvement: Pinpoint specific areas where the search results are lacking or could be enhanced.
        Iterate Search: If necessary, perform additional searches based on the refined understanding of the query and previous results.
        Refine and Synthesize: Combine the results from multiple iterations into a comprehensive and well-rounded response.

Example Workflow:

User Query: "What are the best programming languages for AI development in 2024?"
Self-Define Phase:
    Understand the Query: Identify key components—"best programming languages," "AI development," "2024."
    Formulate Search Plans: Plan multiple searches such as:
        Search 1: "Top programming languages for AI 2024"
        Search 2: "Best languages for AI development 2024"
        Search 3: "Programming languages trends AI 2024"
    Execute Searches: Conduct the planned searches.
Self-Refine Phase:
    Evaluate Results: Review the search outcomes to ensure all key components are covered.
    Identify Gaps: If certain aspects like future trends or expert opinions are missing, plan additional searches.
    Iterate Search: Perform additional searches as needed.
    Refine and Synthesize: Combine the refined search results into a comprehensive response.

Possible Benefits: The copilot is doing exactly what a copilot is intended to do to copilot the search with you suggesting possible searches that would solve the search problem. The idea is to leverage together 2 recent AI prompt research techniques: Self-Discover: Large Language Models Self-Compose Reasoning Structures: https://arxiv.org/abs/2402.03620 Self-Refine: Iterative Refinement with Self-Feedback: https://arxiv.org/abs/2303.17651

Enhanced Accuracy: By iteratively refining the search process, Perplexica Copilot should ensure higher accuracy and relevance of search results. There is research showing that LLM are not currently able to truly self refine, but we try to push the limits as hard as possible. Large Language Models Cannot Self-Correct Reasoning Yet: https://arxiv.org/abs/2310.01798 Comprehensive Coverage: Multiple searches and iterative refinement guarantee that all aspects of the query are thoroughly addressed. User Satisfaction: Users receive well-rounded and informative responses, improving their overall experience.

Conclusion: Perplexica Copilot can leverage advanced AI techniques to enhance search capabilities, providing users with accurate, comprehensive, and relevant information. By incorporating self-define and self-refine phases, the Copilot ensures continuous improvement in search outcomes.

[Zirgite]

const basicSearchRetrieverPrompt = `
You are Perplexica Copilot, an AI designed to enhance web searches by understanding and refining user queries.

### Self-Define Phase:
1. **Understand the Query:**
   - Identify key components of the query.

2. **Formulate Search Plans:**
   - Plan multiple searches to cover different aspects of the query.

3. **Execute Searches:**
   - Conduct the planned searches.

Example:
1. Understand the Query :[Find the best programming languages for AI development in 2024.]
   - Identify key components—"best programming languages," "AI development," "2024."

2. Formulate Search Plans:
   - Search 1: "Top programming languages for AI 2024"
   - Search 2: "Best languages for AI development 2024"
   - Search 3: "Programming languages trends AI 2024"

3. Execute Searches:
   - Conduct the planned searches.

Conversation:
{chat_history}

Follow-up question: {query}
Rephrased question:
`;

[Zirgite]

const basicWebSearchResponsePrompt = `
You are Perplexica Copilot, an AI model who is expert at searching the web and answering user's queries.

### Self-Refine Phase:

1. **Evaluate Initial Results:**
   - Analyze the search results for relevance, accuracy, and completeness.

2. **Identify Areas for Improvement:**
   - Pinpoint specific areas where the search results are lacking or could be enhanced.

3. **Iterate Search:**
   - Perform additional searches if necessary, based on the refined understanding of the query and previous results.

4. **Refine and Synthesize:**
   - Combine the results from multiple iterations into a comprehensive and well-rounded response.

### Generate a response that is informative and relevant to the user's query based on the provided context:
- Use the search results containing a brief description of the content of each page.
- Your response should be unbiased and journalistic in tone.
- Do not repeat the text.

### Guidelines:
- Do not direct the user to open any link or visit any website to get the answer. Provide the answer in the response itself. If the user asks for links, you can provide them.
- Responses should be medium to long in length, informative, and relevant to the user's query.
- Use markdowns to format your response. Use bullet points to list the information.
- Make sure the answer is not short and is informative.
- Cite the answer using [number] notation. Cite each part of the answer so the user can know where the information is coming from.
- Place citations at the end of the relevant sentence. Cite the same sentence multiple times if needed with different numbers.

<context>
{context}
</context>

If you think there's nothing relevant in the search results, say 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from a search engine and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;

[Zirgite]

I tried to make some modifications to the file webSearchAgent. By doing this I approached, sometimes exceeded the performance of copilot in Perplexity. However I used the local model of Microsoft Phi-3 Medium. I made some blind tests asking GPT4 to be the judge of performance. This new prompt forced the answer to be more in depth and more academic.

Also I change the prompt in the web agent Sugestion generation agent, using self-refine approach. The result is more professional suggestions, while the vanilla goes towards mixed bag of theoretical and practical. Also I think for the pro suggestion the number of 5 is low, it has to be 12 as those are pretty narrow.

Bear in mind that there are limitations of the search engine and the results regarding current events are somehow lacking sometimes.

You need to update the file webSearchAgent in Perplexica/src/agents Please keep a copy of the original :). Actually the more in depth does not replace the vanilla version, it is just more in depth

import { BaseMessage } from '@langchain/core/messages';
import {
  PromptTemplate,
  ChatPromptTemplate,
  MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
  RunnableSequence,
  RunnableMap,
  RunnableLambda,
} from '@langchain/core/runnables';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { Document } from '@langchain/core/documents';
import { searchSearxng } from '../lib/searxng';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import type { BaseChatModel } from '@langchain/core/language_models/chat_models';
import type { Embeddings } from '@langchain/core/embeddings';
import formatChatHistoryAsString from '../utils/formatHistory';
import eventEmitter from 'events';
import computeSimilarity from '../utils/computeSimilarity';
import logger from '../utils/logger';

const basicSearchRetrieverPrompt = `
You are Perplexica Copilot, an AI designed to enhance web searches by understanding and refining user queries.

### Self-Define Phase:
1. **Understand the Query:**
   - Identify key components of the query.

2. **Formulate Search Plans:**
   - Plan multiple searches to cover different aspects of the query.

3. **Execute Searches:**
   - Conduct the planned searches.

Example:
User Query:
- Find the best programming languages for AI development in 2024.
1. Understand the Query:
   - Identify key components—"best programming languages," "AI development," "2024."

2. Formulate Search Plans:
   - Search 1: "Top programming languages for AI 2024"
   - Search 2: "Best languages for AI development 2024"
   - Search 3: "Programming languages trends AI 2024"

3. Execute Searches:
   - Conduct the planned searches.

Conversation:
{chat_history}

Follow-up question: {query}
Rephrased question:
`;

const basicWebSearchResponsePrompt = `
You are Perplexica Copilot, an AI model who is expert at searching the web and answering user's queries.

### Self-Refine Phase:
1. **Evaluate Initial Results:**
   - Analyze the search results for relevance, accuracy, and completeness.

2. **Identify Areas for Improvement:**
   - Pinpoint specific areas where the search results are lacking or could be enhanced.

3. **Iterate Search:**
   - Perform additional searches if necessary, based on the refined understanding of the query and previous results.

4. **Refine and Synthesize:**
   - Combine the results from multiple iterations into a comprehensive and well-rounded response.

### Generate a response that is informative and relevant to the user's query based on the provided context:
- Use the search results containing a brief description of the content of each page.
- Your response should be unbiased and journalistic in tone.
- Do not repeat the text.

### Guidelines:
- Do not direct the user to open any link or visit any website to get the answer. Provide the answer in the response itself. If the user asks for links, you can provide them.
- Responses should be medium to long in length, informative, and relevant to the user's query.
- Use markdowns to format your response. Use bullet points to list the information.
- Make sure the answer is not short and is informative.
- Cite the answer using [number] notation. Cite each part of the answer so the user can know where the information is coming from.
- Place citations at the end of the relevant sentence. Cite the same sentence multiple times if needed with different numbers.

<context>
{context}
</context>

If you think there's nothing relevant in the search results, say 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from a search engine and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;

const strParser = new StringOutputParser();

const handleStream = async (
  stream: AsyncGenerator<StreamEvent, any, unknown>,
  emitter: eventEmitter,
) => {
  for await (const event of stream) {
    if (
      event.event === 'on_chain_end' &&
      event.name === 'FinalSourceRetriever'
    ) {
      emitter.emit(
        'data',
        JSON.stringify({ type: 'sources', data: event.data.output }),
      );
    }
    if (
      event.event === 'on_chain_stream' &&
      event.name === 'FinalResponseGenerator'
    ) {
      emitter.emit(
        'data',
        JSON.stringify({ type: 'response', data: event.data.chunk }),
      );
    }
    if (
      event.event === 'on_chain_end' &&
      event.name === 'FinalResponseGenerator'
    ) {
      emitter.emit('end');
    }
  }
};

type BasicChainInput = {
  chat_history: BaseMessage[];
  query: string;
};

const createBasicWebSearchRetrieverChain = (llm: BaseChatModel) => {
  const runSearch = async (input) => {
    if (input === 'not_needed') {
      return { query: '', docs: [] };
    }

    const res = await searchSearxng(input, {
      language: 'en',
    });

    const documents = res.results.map(
      (result) =>
        new Document({
          pageContent: result.content,
          metadata: {
            title: result.title,
            url: result.url,
            ...(result.img_src && { img_src: result.img_src }),
          },
        }),
    );

    return { query: input, docs: documents };
  };

  const iterateSearch = async (input, maxIterations = 3) => {
    let iteration = 0;
    let result = await runSearch(input);

    while (iteration < maxIterations) {
      const refinedQuery = await llm.predict(`Evaluate the following results and suggest how to improve the search results: ${JSON.stringify(result.docs)}`);
      const refinedResult = await runSearch(refinedQuery);
      result.docs = result.docs.concat(refinedResult.docs);
      iteration++;
    }

    return result;
  };

  return RunnableSequence.from([
    PromptTemplate.fromTemplate(basicSearchRetrieverPrompt),
    llm,
    strParser,
    RunnableLambda.from(input => iterateSearch(input.query))
  ]);
};

const createBasicWebSearchAnsweringChain = (
  llm: BaseChatModel,
  embeddings: Embeddings,
) => {
  const basicWebSearchRetrieverChain = createBasicWebSearchRetrieverChain(llm);

  const processDocs = async (docs: Document[]) => {
    return docs
      .map((_, index) => `${index + 1}. ${docs[index].pageContent}`)
      .join('\n');
  };

  const rerankDocs = async ({
    query,
    docs,
  }: {
    query: string;
    docs: Document[];
  }) => {
    if (docs.length === 0) {
      return docs;
    }

    const docsWithContent = docs.filter(
      (doc) => doc.pageContent && doc.pageContent.length > 0,
    );

    const [docEmbeddings, queryEmbedding] = await Promise.all([
      embeddings.embedDocuments(docsWithContent.map((doc) => doc.pageContent)),
      embeddings.embedQuery(query),
    ]);

    const similarity = docEmbeddings.map((docEmbedding, i) => {
      const sim = computeSimilarity(queryEmbedding, docEmbedding);

      return {
        index: i,
        similarity: sim,
      };
    });

    const sortedDocs = similarity
      .sort((a, b) => b.similarity - a.similarity)
      .filter((sim) => sim.similarity > 0.5)
      .slice(0, 15)
      .map((sim) => docsWithContent[sim.index]);

    return sortedDocs;
  };

  return RunnableSequence.from([
    RunnableMap.from({
      query: (input: BasicChainInput) => input.query,
      chat_history: (input: BasicChainInput) => input.chat_history,
      context: RunnableSequence.from([
        (input) => ({
          query: input.query,
          chat_history: formatChatHistoryAsString(input.chat_history),
        }),
        basicWebSearchRetrieverChain
          .pipe(rerankDocs)
          .withConfig({
            runName: 'FinalSourceRetriever',
          })
          .pipe(processDocs),
      ]),
    }),
    ChatPromptTemplate.fromMessages([
      ['system', basicWebSearchResponsePrompt],
      new MessagesPlaceholder('chat_history'),
      ['user', '{query}'],
    ]),
    llm,
    strParser,
  ]).withConfig({
    runName: 'FinalResponseGenerator',
  });
};

const basicWebSearch = (
  query: string,
  history: BaseMessage[],
  llm: BaseChatModel,
  embeddings: Embeddings,
) => {
  const emitter = new eventEmitter();

  try {
    const basicWebSearchAnsweringChain = createBasicWebSearchAnsweringChain(
      llm,
      embeddings,
    );

    const stream = basicWebSearchAnsweringChain.streamEvents(
      {
        chat_history: history,
        query: query,
      },
      {
        version: 'v1',
      },
    );

    handleStream(stream, emitter);
  } catch (err) {
    emitter.emit(
      'error',
      JSON.stringify({ data: 'An error has occurred please try again later' }),
    );
    logger.error(`Error in websearch: ${err}`);
  }

  return emitter;
};

const handleWebSearch = (
  message: string,
  history: BaseMessage[],
  llm: BaseChatModel,
  embeddings: Embeddings,
) => {
  const emitter = basicWebSearch(message, history, llm, embeddings);
  return emitter;
};

export default handleWebSearch;

[Zirgite]

What are effective strategies for managing stress?

---

Perplexica with new prompt for more advanced response.

Effective strategies for managing stress involve a variety of mental, emotional, and behavioral techniques that can help reduce its negative impacts on well-being. Here's an overview:

1. Time Management [8]: Organizing tasks, setting appropriate goals, and using tools to maintain control over time spent on various activities is integral for reducing stress levels. Effective time management ensures that workloads are manageable, leaving more room for relaxation and enjoyment of life outside of work responsibilities.

2. Physical Activity [6]: Regular exercise has numerous benefits in combating stress. It releases endorphins (feel-good hormones) while also reducing blood pressure and the levels of stress hormones like norepinephrine, which can help create a sense of calmness and relaxation.

3. Mindfulness Techniques [11]: Incorporating mindfulness into daily routines is an effective way to manage stress. Deep breathing exercises, meditation, or yoga are examples of practices that help keep the mind focused on the present moment and can assist in managing anxiety levels.

4. Hugging [2]: Surprisingly, hugging is an effective way to combat stress because it releases oxytocin (the "love hormone") which promotes bonding while also reducing blood pressure and the stress hormone norepinephrine. This activation of relaxation response helps calm you quickly.

5. Healthy Diet [3]: Consuming a balanced diet can help combat stress, as it provides essential nutrients to support overall health and maintain energy levels throughout the day. Avoiding caffeine is also recommended because it can compound the effects of stress on the body.

6. Coping Strategies [9]: Everyone responds differently to stress, so finding what works best for you may require experimentation with various techniques and strategies. Mental, emotional, and behavioral tools like journaling, seeking social support, or engaging in hobbies can contribute significantly towards reducing overall stress levels.

7. Stress Management Programs [5]: Stress management programs offered by reputable institutions (such as Harvard Medical School) provide a wealth of proven techniques that help individuals combat the damaging effects of stress and build resilience. These may include cognitive-behavioral therapy, relaxation training, or problem-solving skills training.

8. Self-Healing [5]: By calling upon your body's rich potential for self-healing through various practices like meditation, exercise, and proper nutrition, you can manage stress more effectively while also promoting overall wellness.

9. Workplace Stress Management [10]: Workplaces that implement stress management mechanisms such as employee assistance programs, mindfulness activities, or relaxation zones provide employees with the resources to cope better with work-related challenges and maintain a healthy balance between personal life and career demands.

Remember, finding the best techniques for managing stress may take some experimenting and practice, but it's essential to keep looking for tools that will help you manage life's inevitable ups and downs in a healthy way [15].

Citations: [1] https://positivepsychology.com/stress-management-techniques-tips-burn-out/ [2] https://www.betterup.com/blog/stress-management-techniques [3] https://health.clevelandclinic.org/how-to-relieve-stress [4] https://www.mayoclinic.org/tests-procedures/stress-management/about/pac-20384898 [5] https://www.health.harvard.edu/mind-and-mood/stress-management-enhance-your-well-being-by-reducing-stress-and-building-resilience [6] https://www.helpguide.org/articles/stress/stress-management.htm [7] https://www.nature.com/articles/s41598-024-66357-7 [8] https://www.msn.com/en-us/health/other/15-ways-to-decrease-stress-simple-strategies-for-a-calmer-life/ar-BB1hGJ0k [9] https://www.berkeleywellbeing.com/stress-management.html [10] https://positivepsychology.com/workplace-stress-management/ [11] https://www.launchnotes.com/blog/effective-stress-management-strategies-a-comprehensive-guide [12] https://my.clevelandclinic.org/health/articles/8133-stress-10-ways-to-ease-stress [13] https://thriveglobal.com/stories/7-strategies-for-effective-stress-management/ [14] https://positivepsychology.com/cope-with-stress/ [15] https://www.verywellmind.com/tips-to-reduce-stress-3145195

Related How can I incorporate mindfulness techniques into my daily routine? What are some examples of stress management programs offered by reputable institutions like Harvard Medical School? Can you provide more information on the benefits of regular physical activity in combating stress? Are there any specific dietary changes that can help reduce stress levels? How do workplaces implement effective stress management mechanisms for their employees?

Test 2 Vanilla Perplexity

Here are some effective strategies for managing stress:

Identify the sources of stress

• Look at the big picture and ask yourself how important the stressful situation will be in the long run[1]
• Adjust your standards and stop setting yourself up for failure by demanding perfection[1]
• Practice gratitude by reflecting on the things you appreciate in your life[1]

Accept what you cannot control

• Don't try to control the uncontrollable, like the behavior of other people[1]
• Look for the upside and see challenges as opportunities for personal growth[1]
• Learn to forgive and let go of anger and resentments[1]

Prioritize self-care

• Establish a self-care routine that includes activities that bring joy, relaxation, and rejuvenation[3]
• Get enough sleep, eat a balanced diet, and practice good self-care habits[3]
• Make time for hobbies and interests that you enjoy[2]

Manage your time effectively

• Prioritize tasks and tackle the most important ones first[1]
• Use time blocking to allocate specific time slots for different tasks throughout your day[3]
• Avoid multitasking, which can lead to decreased productivity and increased stress[3]

Practice relaxation techniques

• Exercise regularly through activities like walking, running, yoga, or tai chi[1][2]
• Practice meditation, which can lower stress, anxiety, and chronic pain[2]
• Use deep breathing, counting to 10 slowly, and positive self-talk[2][4]

Seek support

• Share your feelings with a trusted friend or family member[1]
• Consider therapy or counseling to learn healthy ways of dealing with stress[2]
• Get involved in your community through volunteering or other activities[4]

Remember, managing stress is about taking charge of your thoughts, emotions, schedule, and environment. By experimenting with different strategies and finding what works best for you, you can improve your overall well-being and resilience in the face of life's challenges.

Citations: [1] https://www.helpguide.org/articles/stress/stress-management.htm [2] https://www.webmd.com/balance/stress-management/stress-management [3] https://www.movebeyond.com.au/10-effective-strategies-for-stress-management/ [4] https://med.libretexts.org/Bookshelves/Health_and_Fitness/Concepts_of_Fitness_and_Wellness_%28Flynn_et_al.%29/09:_Stress/9.02:_What_Are_the_Strategies_for_Managing_Stress [5] https://www.mentalhealth.org.uk/explore-mental-health/publications/how-manage-and-reduce-stress

What are some quick stress-relief techniques I can use at work How can I identify the main sources of stress in my life What role does gratitude play in managing stress How can I balance my schedule to reduce stress What are the benefits of practicing forgiveness in stress management

Test 3: Gemini with gmail check

Stress management is crucial for maintaining overall well-being. Here are some effective strategies you can try: Relaxation Techniques: • Deep breathing: This simple yet powerful technique can significantly reduce stress. Breathe in slowly for 4 seconds, hold for 7 seconds, and exhale slowly for 8 seconds. Repeat for a few minutes. • Meditation: Mindfulness meditation helps you focus on the present moment and let go of worries. There are many guided meditations available online or in apps. • Progressive muscle relaxation: Tense and relax different muscle groups one by one to release tension throughout your body. Lifestyle Changes: • Exercise regularly: Physical activity is a great stress reliever. Aim for at least 30 minutes of moderate-intensity exercise most days of the week. • Healthy diet: Eating nutritious foods provides your body with the energy it needs to cope with stress. Limit processed foods, sugar, and caffeine. • Quality sleep: Aim for 7-8 hours of quality sleep each night. Develop a relaxing bedtime routine to wind down before sleep. • Time management: Feeling overwhelmed by tasks can be stressful. Prioritize, set realistic goals, and delegate tasks when possible. Mindset Shifts: • Positive attitude: A positive outlook can help you cope with stressful situations better. • Acceptance: Learn to accept things you cannot control. Focus on what you can influence and let go of the rest. • Set boundaries: Learn to say no to requests that would add unnecessary stress to your life. Additional Techniques: • Spend time in nature: Take a walk in the park or spend time outdoors to reduce stress and improve mood. • Connect with loved ones: Strong social connections provide support and help you feel less alone. • Hobbies and relaxation: Make time for activities you enjoy, whether it's reading, listening to music, or spending time on hobbies. Remember, what works for one person might not work for another. Experiment and find the stress management techniques that work best for you. If you're feeling overwhelmed by stress, consider talking to a therapist or counselor for additional support.

More Effective strategies for managing stress Whare are 4 ways to manage stress effectively What are 5 stress management strategies

[Zirgite]

The results rated by gpt 4o. Perplexica (Phi 3 Medium with ollama) outperforms overall both vanilla Perplexity and google gemini. Overall Evaluation

Best Response (Content): Test 1: Perplexica with new prompt. It offers the most comprehensive and detailed list of strategies, covering a wide range of techniques from physical activities to workplace strategies. It is thorough and includes numerous citations for further reading.
Best Response (Organization): Test 2 Perplexity vanilla. The structure into clear categories makes it very user-friendly and easy to understand different aspects of stress management.
Best Response (Citation Quality): Test 1 Perplexica with new prompt. Provides the most extensive references, which adds credibility and allows users to explore the topic further.
however Concise and Practical Response: Test 3 Gemini. While lacking in citations, it presents the strategies in a straightforward and practical manner.

[Ed-cred]

This seems like a very well documented attempt @Zirgite, thank you for all of your answers i really appreciate your thoroughness. However, when i tried the example you provided it triggers an exceptions in the cosine-similarity calculation function, see below. I suspect this could either be improper handling of async data calls, but i have not found the fix for my issue yet so i am unable to test your findings. I am also using Ollama btw with llama3:8b-instruct-q8.

node:internal/process/promises:391
    triggerUncaughtException(err, true /* fromPromise */);
    ^

Error: cosine-similarity()::invalid input argument. Input arrays must have the same length.
    at similarity (/home/perplexica/node_modules/compute-cosine-similarity/lib/index.js:52:9)
    at computeSimilarity (/home/perplexica/dist/utils/computeSimilarity.js:12:56)
    at /home/perplexica/dist/agents/copilotWebSearchAgent.js:154:57
    at Array.map (<anonymous>)
    at RunnableLambda.rerankDocs [as func] (/home/perplexica/dist/agents/copilotWebSearchAgent.js:153:42)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /home/perplexica/node_modules/@langchain/core/dist/runnables/base.cjs:1419:33

If the results of your tests are indeed valid, this would be a great improvement for a user looking for detailed information on a topic or running a more complex query, however there would be a need to abstract this away since from my understanding copilot mode needs to be a mode for Perplexica to function that is Agent agnostic.

[Zirgite]

I cannot upload the files. So the easy thing is just to change the file file webSearchAgent.ts in Perplexica/src/agents In the file change only the prompt part: const basicWebSearchResponsePrompt =PUT PROMPT THERE; const basicWebSearchResponsePrompt = PUT PROMPT THERE; Those prompts are detailed above. We do not touch the code. In the code above I tried to make iterative search with const iterateSearch. But this is just as conceptual idea. The idea of the copilot is that the ai to understand the search and make a plan to conduct several lateral searches and then to combine the answer.

[Zirgite]

Also further in the line i made a prompt for the suggestionGeneratorAgent. This tries to make more professional and targetted suggestions for next search. Those suggestions dig deeper into the topic, and I wish to have at least 8 to 12, as they are narrow and not broad like the the vanilla. This does not replace the vanilla, as the vanilla is more friendly broad search suggestion. It is truly different mode of operation.

const suggestionGeneratorPrompt = `
You are an AI suggestion generator for an AI powered search engine. You will be given a conversation below. You need to generate 4-5 suggestions based on the conversation. The suggestions should be relevant to the conversation and helpful to the user. 
Keep a note that the user might use these suggestions to ask a chat model for more information. 

### Self-Define Phase:
1. **Generate Initial Suggestions:**
   - Create 4-5 suggestions based on the conversation.

### Self-Refine Phase:
2. **Evaluate Initial Suggestions:**
   - Analyze the suggestions for relevance, informativeness, and clarity.
3. **Identify Areas for Improvement:**
   - Pinpoint specific areas where the suggestions could be enhanced.
4. **Refine Suggestions:**
   - Improve the suggestions based on the evaluation.

Provide the refined suggestions separated by newlines between the XML tags <suggestions> and </suggestions>. For example:

<suggestions>
Tell me more about SpaceX and their recent projects
What is the latest news on SpaceX?
Who is the CEO of SpaceX?
</suggestions>

Conversation:
{chat_history}
`;

[Zirgite]

Another test with phi-3 model comparing Perplexica with deeper implementations copilot and Perplexity showed, that the perplexica is diving deeper in the subject compared to copilot perplexity, also the selected links are more into big dive into the topic. While Perplexity copilot remains more easier to grasp. test: How does photosynthesis work in plants?

Test 1: Perplexica with 'copilot' self define and self-refine Photosynthesis is a vital process that allows plants to convert light energy from the sun into chemical energy, which is stored as sugar (glucose) and used for growth. The overall reaction of photosynthesis can be summarized with the equation:

6CO2 + 12H2O + Light Energy → C6H12O6 + 6O2 + 6H2O

The process involves two main stages: light-dependent reactions and light-independent reactions (Calvin cycle). Here's a step-by-step overview of how photosynthesis works in plants:

1. Light absorption: Photosynthesis begins when chlorophyll, the primary pigment found within chloroplasts in plant cells, absorbs sunlight. The energy from light excites electrons in the chlorophyll molecules, initiating a series of reactions known as the light-dependent reactions (context [3][13]).

2. Light-dependent reactions: These take place within the thylakoid membranes of chloroplasts and involve splitting water molecules into oxygen gas, protons, and electrons (context [6][9][14]): a. Water is split by an enzyme called photosystem II, generating oxygen as a waste product that is released into the atmosphere (context [8]). b. The energized electrons from chlorophyll pass through electron transport chain components such as plastoquinone and cytochrome complex. As they move, energy is used to create ATP via chemiosmosis (proton gradient) and reduce NADP+ into NADPH (context [9][10]). c. The light-dependent reactions result in the production of oxygen gas, ATP, and NADPH, which are essential for the next stage of photosynthesis: the Calvin cycle or light-independent reactions (context [3][8][14]).

3. Light-independent reactions (Calvin Cycle): These occur in the stroma surrounding the thylakoid membranes within chlorophrenoids. The ATP and NADPH produced during the light-dependent reactions are utilized to convert CO2 into sugar: a. Carbon fixation: Enzyme RuBisCO catalyzes the reaction between CO2 and ribulose biphosphate (RuBP) in a process known as carbon fixation, resulting in an unstable 6-carbon intermediate that rapidly breaks down into two molecules of 3-phosphoglycerate (3-PGA). b. Reduction: Each 3-PGA is phosphorylated by ATP and reduced by NADPH to produce glyceraldehyde 3-phosphate (G3P), a precursor for sugar synthesis. For every six CO2 molecules fixed, one G3P can exit the cycle as an end product; it takes three turns of the Calvin Cycle to produce one G3P moleccule (context [9][14]). c. Regeneration: The remaining five G3Ps are used in a series of reactions that regenerate RuBP, allowing the cycle to continue (context [8][9]).

In summary, photosynthesis is an essential process for plants and other organisms capable of converting light energy into chemical energy. It involves two main stages: the light-dependent reactions generating ATP and NADPH while producing oxygen as a byproduct; and the Calvin cycle using those molecules to fix carbon dioxide into sugar, which can then be used for growth or stored (context [3][8][14]).

Citations: [1] https://www.nagwa.com/en/videos/627178468285/ [2] https://quizlet.com/100506197/photosynthesis-and-cellular-respiration-flash-cards/ [3] https://bio.libretexts.org/Bookshelves/Botany/Botany_(Ha_Morrow_and_Algiers)/04%3A_Plant_Physiology_and_Regulation/4.01%3A_Photosynthesis_and_Respiration [4] https://quizlet.com/410956704/photosynthesis-and-cellular-respiration-flash-cards/ [5] https://quizlet.com/69179652/photosynthesis-recall-practice-flash-cards/ [6] https://www.livescience.com/51720-photosynthesis.html [7] https://www.khanacademy.org/science/hs-biology-tx/xd86e652aaf556bcf:photosynthesis-and-cellular-respiration [8] https://www.khanacademy.org/science/ap-biology/cellular-energetics/photosynthesis/a/light-dependent-reactions [9] https://testbook.com/objective-questions/mcq-on-photosynthesis--5eea6a0c39140f30f369e042 [10] https://www.bbc.co.uk/bitesize/topics/zvrrd2p/articles/zjqfsk7?course=zdcg3j6 [11] https://www.khanacademy.org/science/biology/photosynthesis-in-plants/photorespiration--c3-c4-cam-plants/a/c3-c4-cam-plants [12] https://phys.org/news/2024-07-photosynthesis-powers-world-fuels-fundamental.html [13] https://www.khanacademy.org/science/biology/photosynthesis-in-plants [14] https://www.nationalgeographic.org/encyclopedia/photosynthesis/ [15] https://www.bbc.co.uk/bitesize/articles/zdxkcmn

Related How do light-dependent reactions contribute to photosynthesis? Can you explain more about the role of chlorophyll in photosynthesis? What is the significance of ATP and NADPH produced during photosynthesis? Could you elaborate on how carbon dioxide is converted into sugar during the Calvin cycle? Are there any factors that can affect the efficiency of photosynthesis in plants?

Test 2: Perplexity with copilot

How Photosynthesis Works in Plants

Photosynthesis is the process by which plants convert light energy from the sun into chemical energy in the form of glucose. This process is essential for plant growth and survival, as well as for supporting many other lifeforms on Earth. Here's a detailed overview of how photosynthesis works in plants:

The Light Reactions

1. Light Absorption: The process begins when sunlight hits the leaves of a plant. The chloroplasts within the plant cells contain chlorophyll, a green pigment that absorbs the red and blue wavelengths of light while reflecting the green wavelengths, which is why plants appear green.
2. Light-Dependent Reactions: The absorbed light energy excites electrons in the chlorophyll molecules, initiating a series of light-dependent chemical reactions. These reactions split water molecules (H2O) into hydrogen ions (H+), electrons, and oxygen gas (O2). The electrons are used to produce two energy-carrying molecules: ATP and NADPH.

The Calvin Cycle

3. Carbon Fixation: The ATP and NADPH produced in the light-dependent reactions are then used in the Calvin cycle, also known as the light-independent or dark reactions. In this stage, the plant uses the energy from ATP and NADPH to convert carbon dioxide (CO2) from the air into organic compounds, primarily glucose (C6H12O6).
4. Glucose Synthesis: The Calvin cycle involves a series of enzymatic reactions that "fix" the carbon from CO2 into 3-carbon compounds. These compounds are then rearranged and combined to form the 6-carbon glucose molecule.

Oxygen Release

5. Oxygen Release: As a byproduct of the light-dependent reactions, oxygen gas (O2) is released from the plant and diffuses out through the stomata (pores) in the leaves. This oxygen is then available for other organisms, such as animals, to use in respiration.

Energy Storage

6. Energy Storage: The glucose produced during photosynthesis is either used immediately by the plant for energy or stored for later use in the form of starch. This stored energy can then be accessed and used by the plant as needed for growth, maintenance, and other vital processes.

In summary, photosynthesis in plants is a complex process that involves the absorption of light energy, the conversion of that energy into chemical energy in the form of ATP and NADPH, the fixation of carbon dioxide into organic compounds, and the release of oxygen as a byproduct. This process is essential for the survival and growth of plants, as well as for supporting the broader ecosystem on Earth.[1][2][3][4][5]

Citations: [1] https://www.snexplores.org/article/explainer-how-photosynthesis-works [2] https://www.britannica.com/science/photosynthesis [3] https://ssec.si.edu/stemvisions-blog/what-photosynthesis [4] https://www.sciencefocus.com/nature/how-does-photosynthesis-work [5] https://education.nationalgeographic.org/resource/photosynthesis/ [6] https://en.wikipedia.org/wiki/Photosynthesis [7] https://www.life.illinois.edu/govindjee/encyc/encarta.htm [8] https://microbenotes.com/photosynthesis/ [9] https://www.khanacademy.org/science/ap-biology/cellular-energetics/photosynthesis/a/intro-to-photosynthesis [10] https://deepgreenpermaculture.com/2023/12/04/understanding-how-photosynthesis-works-a-comprehensive-guide-for-gardeners/ [11] https://bio.libretexts.org/Courses/Lumen_Learning/Biology_for_Non-Majors_I_%28Lumen%29/06:_Metabolic_Pathways/6.06:_Photosynthesis [12] https://www.youtube.com/watch?v=UPBMG5EYydo [13] https://online.kidsdiscover.com/unit/plants/video/how-photosynthesis-works [14] https://www.bbc.co.uk/bitesize/guides/zcktw6f/revision/3 [15] https://www.bbc.co.uk/bitesize/articles/zdxkcmn

What role do chloroplasts play in photosynthesis How do plants use ATP and NADPH in the Calvin cycle What is the significance of rubisco in photosynthesis How do light-dependent reactions differ from light-independent reactions Why do plants reflect green light instead of absorbing it

gpt4o arbiter: To evaluate the two AI-generated responses on the query "How does photosynthesis work in plants?" and the related searches proposed, we need to examine the details of each answer, the depth and accuracy of the information provided, and the quality and relevance of the sources cited. Detailed Comparison: Test 1:

Structure and Detail:
    Comprehensive Coverage: The answer covers the entire process of photosynthesis in two main stages: light-dependent reactions and light-independent reactions (Calvin cycle).
    Step-by-Step Explanation:
        Light Absorption: Chlorophyll absorbs sunlight, exciting electrons.
        Light-Dependent Reactions: Involves splitting water molecules, generating oxygen, ATP, and NADPH. Detailed components like photosystem II, plastoquinone, and cytochrome complex are mentioned.
        Calvin Cycle: Explains carbon fixation, reduction, and regeneration. Includes enzyme RuBisCO and specifics about intermediate molecules like 3-phosphoglycerate and glyceraldehyde 3-phosphate.

Citations and Links:
    Sources: Cites 15 sources including educational websites (Khan Academy, BBC Bitesize), scientific explanations (National Geographic), and various educational platforms (Quizlet).
    Quality: The sources are reliable, well-regarded in educational contexts, and provide in-depth information on photosynthesis.

Related Searches:
    The proposed related searches are directly relevant to photosynthesis, aiming to deepen the understanding of specific aspects like light-dependent reactions, chlorophyll's role, ATP and NADPH significance, and factors affecting photosynthesis efficiency.

Test 2:

Structure and Detail:
    Simplified Explanation: Provides a clear and concise overview of photosynthesis.
    Step-by-Step Explanation:
        Light Absorption: Chloroplasts absorb sunlight using chlorophyll.
        Light-Dependent Reactions: Describes the splitting of water molecules and production of ATP and NADPH without diving into the detailed components and processes involved.
        Calvin Cycle: General description of carbon fixation and glucose synthesis.
        Oxygen Release: Mentions oxygen release as a byproduct.
        Energy Storage: Notes how glucose is stored as starch or used immediately.

Citations and Links:
    Sources: Cites 15 sources including reputable scientific websites (Britannica, National Geographic), educational platforms (Khan Academy), and specialized articles (ScienceFocus, MicrobeNotes).
    Quality: The sources are reliable and provide a good general understanding of photosynthesis, but may lack the depth found in Test 1.

Related Searches:
    The proposed related searches are relevant and aim to explain specific elements of photosynthesis, such as the role of chloroplasts, the usage of ATP and NADPH, rubisco's significance, and differences between light-dependent and light-independent reactions.

Evaluation of Sources: Test 1:

Depth: Sources like Khan Academy and National Geographic provide detailed educational content suitable for deep understanding.
Reliability: Cites established educational platforms and scientific explanations, ensuring credibility.
Contextual Citations: References are provided for specific points within the explanation, enhancing the credibility and allowing for easy verification of facts.

Test 2:

Depth: Sources such as Britannica and Khan Academy offer reliable, general overviews but with less technical detail compared to Test 1.
Reliability: Uses well-known, authoritative sources, ensuring trustworthy information.
Broad Coverage: The sources cover a broad spectrum of educational materials, which can help in understanding the basics but might miss finer details.

Quality of Related Searches: Test 1:

Relevance: Directly related to the specific processes and components of photosynthesis.
Depth: Encourages exploring deeper aspects of the photosynthetic process, enhancing understanding.

Test 2:

Relevance: Addresses fundamental questions about photosynthesis and its components.
Accessibility: Aims to clarify basic concepts and functions within the photosynthetic process.

Conclusion:

Test 1: Provides a more detailed, technical, and step-by-step explanation of photosynthesis, supported by numerous high-quality sources. The related searches aim to deepen understanding of specific aspects.
Test 2: Offers a clear, concise, and general overview suitable for a basic understanding, supported by authoritative sources. The related searches focus on explaining fundamental concepts.

In summary, Test 1 Perplexica is more suitable for readers seeking a detailed, in-depth understanding of photosynthesis, while Test 2 Perplexity with copilot is ideal for those needing a clear and straightforward overview. Both answers are well-supported by reliable sources, but the depth and technical detail in Test 1 Perplexica provide a more comprehensive explanation.

[Ed-cred]

I see, i was particularily interested in the iterative search part, because although we sacrifice response time a model being able to self calibrate the query based on the responses and sources it finds seems like epitome of the Copilot mode. I was able to test some searches using our updated prompts and i could see that the answers provided more detail with the model offering more descriptive information about the things it found related to the query. This was not a very scientific test, i basically asked the initial web search agent and the one updated with the new prompts provided the same question and observed a difference in the depth of the information provided in the results. This seems like a good stepping stone to achieving the Copilot functionality described in the Perplexica readme, but the second part, namely the Copilot mode being able to access web pages and understand their content seems more akin to something like an agent chain might be able to provide. From what i have read langchain, the main library currently being used to power Perplexica functionality offers something called langgraph that enables creating such agent chains, enabling action orchestration and other such actions necessary for successfully handling the more complex queries.

This is why i was initially asking if this feature was being actively developed, or if not what direction the project owner would like to take since things like the prompt engineering that you have showcased can go a long way in focusing the results of the model, but other steps would most likely need to be taken in order to achieve the functionality currently being described in the project readme.

[Zirgite]

Thanks for the input. Right now it is toe to toe with Perplexity copilot. The perplexity focus more to be more usefriendly, why Perplexica with updated prompt makes pro searches and the results are for specialists and are less for the general audience looking for easy to understand results. The perplexica vanilla is tuned to be user friendly, while this prompt is for nose dive into the topic.

[Zirgite]

I think I made a mistake yesterday as I think there is need to run the docker install again to intorduce the new agent into the docker. Please correct me if I am wrong. This is what I made today. The results are really good. When I asked to be simple it is simple when I ask to be complicated it is complicated and in more depth. I made several benchmarks. All are toe to toe, it is interesting that Perplexity vanilla on some occasions gives better results. than the Copilot Perplexity. The new prompt is sometimes better than perplexity. But they are very very close. However now with the new update there are no references from where it gets the information.

[Ed-cred]

The search seems to be performing better when using these new prompts, i know not very scientific but i have not had the time to properly test it like you have done. I have not had the problem of no references, but i did experience in the case of some more niche searches on topics that are less popular the references it gives for context are nonsensical and have nothing to do with the actual text, even if the generated response seems sound. This seems to be an issue where the model outputs a reference saying it found certain information in there, but the link it provides is completely unrelated so fact checking this information becomes more difficult.

[Zirgite]

I update the prompt here: https://github.com/ItzCrazyKns/Perplexica/discussions/266

The idea is to follow the logic of Perplexica as it is. I made some wrong assumptions, the code above is just idea how to make a new function iterative search that is interesting for copilot functionality. Here instead to make an iterative search i make just bigger search looking with neighboring key words.

const basicSearchRetrieverPrompt --- just formulated the search query

const basicWebSearchResponsePrompt --- formulates the answer

I made an evaluation prompt (also in the discussion) to compare with another AI (gpt 4o) as judge the performance between Perplexity and Perplexica.

Some interesting finds:

1. Do not stick with the first answer, reiterate the same search it can come better.
2. You still can guide the model in the query if you want a simpler, or detailed answer
3. Do not use perplexica for current events, news, local weather etc. Perplexica does not go in the feature latest in the searxng

ideas for later work: I may try to put the criteria in the prompt to evaluate: -Relevance: How accurately does the AI model interpret and respond to the query? -Clarity: How clear and understandable are the AI-generated responses? -Helpfulness: How useful are the AI responses in guiding the user to relevant information? -User Experience: How intuitive and seamless is the interaction with the AI model?

[Ed-cred]

How are the sources for this version of the prompt for you, cause you previously stated that they were absent from the generated response. In my experience testing it today, it seems that it is able to give some ballpark references at the bottom of the generated text, but the sources listed at the top of the query, the indexed ones, are still way off the mark, and usually they direct to some websites or articles about rephrasing questions, or unrelated topics to the query. Have you had similar experiences with the sources stated at the top of the answer generated?

[Zirgite]

This is my latest version.But before that Try different embedding as they make the model to do irrelevant searches. And when the searches are irrelevant I think the model starts to use its own knowledge. For me emeddings ollama with emmedings model llama 3 works the best.

const basicSearchRetrieverPrompt = `
You will be given a conversation below and a follow-up question. Your task is to rephrase the follow-up question if needed so it can be used as a standalone query for web searching.
If it is a writing task or a simple greeting rather than a question, return \`not_needed\`.

### Self-Define Phase:
1. **Understand the Query:**
   - Identify key components of the query.

2. **Formulate Rephrased Questions:**
   - Create multiple rephrased questions that cover different aspects of the original query.

3. **Finalize Rephrased Question:**
   - Combine the rephrased questions into a single, comprehensive search query.

Example:
User Query:
- Find the best programming languages for AI development in 2024.
1. Understand the Query:
   - Key components: "best programming languages," "AI development," "2024."

2. Formulate Rephrased Questions:
   ["Top programming languages for AI 2024", "Best languages for AI development 2024", "Programming languages trends AI 2024"]

3. Finalize Rephrased Question:
   "Top programming languages for AI 2024 + Best languages for AI development 2024 + Programming languages trends AI 2024"

Conversation:
{chat_history}

Follow-up question: {query}
Rephrased question:
`;

const basicWebSearchResponsePrompt = `
You are Perplexica, an AI expert at searching the web and answering users' queries.

### Guidelines for Quality:

1. **Interaction Quality**:
   - **Relevance**: How accurately does the AI model interpret and respond to the query?
   - **Clarity**: How clear and understandable are the AI-generated responses?
   - **Helpfulness**: How useful are the AI responses in guiding the user to relevant information?
   - **User Experience**: How intuitive and seamless is the interaction with the AI model?

2. **Content Relevance**:
   - **Depth**: Does the content provided cover the query comprehensively?
   - **Accuracy**: Is the content factually correct and well-researched?
   - **Authority**: Are the sources of the content reputable and reliable?

### Self-Define Phase:
1. **Understand and Analyze Search Results:**
   - The context consists of search results containing brief descriptions of the content.
   - Analyze the results for relevance, accuracy, and completeness.

2. **Formulate Response Plan:**
   - Use the search results as context.
   - Your response should be unbiased and journalistic in tone.
   - Do not repeat the text.

3. **Write Response:**

   - Always use search results as context for your answer.
   - Generate an informative and relevant response based on the context and use [Guidelines for Quality].
   - Do not direct the user to open any links or visit websites. Provide the answer in the response. If the user asks for links, you can provide them.
   - Responses should be medium to long, informative, and relevant.
   - Use markdown for formatting. Use bullet points for lists.
   - Make sure the answer is informative and well-cited using [number] notation.
   - Place citations at the end of the relevant sentences.

### Self-Refine Phase: This is an internal process, not to be shared with the user.
1. **Evaluate Written Response:**
   - Check if [Guidelines for Quality] have been followed.

2. **Identify Areas for Improvement:**
   - Pinpoint specific areas for enhancement.

3. **Iterate Written Response:**
   - Redact further if necessary based on guidelines and refined understanding.

4. **Refine and Synthesize:**
   - Combine results from multiple iterations into a comprehensive response.

Anything within the \`context\` HTML block is retrieved from a search engine and should be cited but not mentioned explicitly in your response. Today's date is ${new Date().toISOString()}.
<context>
{context}
</context>

If there’s nothing relevant in the search results, you can say, "Hmm, sorry, I could not find any relevant information on this topic. Would you like me to search again or ask something else?".
`;

[Ed-cred]

I personally ended up reverting to a modified version of the regular prompt for more consistent source outputs. While testing your prompt i noticed that while the llm was context aware, in the sense that it was able to specify the correct reference in text form, whenever it came to introducing that reference as a link embedded in the text it usually failed and provided something totally unrelated. I am not sure what other tweaks you have done, particularly to the embeddings side i saw you metion, but that was the main issue for me with indexing the references and sorting their relevance, and often times getting back references that were nonsensical in the text.

[Zirgite]

Debugging an ai + code is cream of the cream of difficulty. You never know where is the problem, in the prompt or in the hard code. I reversed back to the original prompt too to check but I get the same thing sometimes the references are perfectly aligned sometimes not. I went back to my last prompt and it is the same thing sometimes i get perfect reference alignment sometimes I don't. That means the AI prompt is the problem. The reward is big though, as I get to have a deep search engine working on the same level of performance as perplexity copilot. 'I think that Perplexity copilot does not always go to the web content but is just searching with more references that the regular vanilla'. Also I think Perplexity, windows copilot and Gemini are SPECIFICALLY instructed to be user friendly and get easy to understand results. And so when the topic is deep You need to tell them SPECIFICALLY that you are not the average user but a specialist as they do not know. I am happy because all this effort made me understand quite a lot useful finds in ai assisted web search, the others are in the discussion https://github.com/ItzCrazyKns/Perplexica/discussions/266

[Zirgite]

Sometimes I need the vanilla suggestion that work like any other suggestions in Prplexity, Windows copilot, google etc. But sometimes I really want cutting suggestions into the topic.

There is bug that is common with the normal prompt when going deeper into the topic the suggestions break down starting with level 5 of deep following of suggestions. # prompt for suggestionGeneratorAgent

const suggestionGeneratorPrompt = `
You are an AI suggestion generator for an AI-powered search engine. You will be given a conversation below. Generate 4-5 suggestions based on the conversation.  
Keep a note that the user might use these suggestions to ask a chat model for more information.

### Guidelines for Quality of the suggestions:

1. **Relevance**: How relevant are the suggestions  to the original query?
2. **Diversity**: Do the related suggestions cover a wide range of subtopics related to the original or previous query?
3. **Helpfulness**: Are the related suggestions useful in refining or expanding the search?
4. **Appropriate Length**: Are the suggestions of suitable length to be informative and useful without being too brief or too verbose?

### Self-Define Phase:

1. **Understand and Analyze the Conversation**:
   - Review the context of the conversation below.

2. **Generate Initial Suggestions**:
   - Follow the  [Guidelines for Quality of the suggestions]
   - Create 4-5 suggestions based on the conversation.

### Self-Refine Phase (Internal Process):

3. **Evaluate Initial Suggestions**:
   - Analyze if the suggestions follow the  [Guidelines for Quality of the suggestions].

4. **Identify Areas for Improvement**:
   - Pinpoint specific areas for enhancement.

5. **Refine Suggestions**:
   - Make further refinements if necessary based on guidelines and refined understanding.

6. **Refine and Synthesize**:
   - Combine results from multiple iterations into comprehensive suggestions.

### Output Format:

Provide these suggestions separated by newlines between the XML tags <suggestions> and </suggestions>. For example:

<suggestions>
Tell me more about SpaceX and their recent projects.
What is the latest news on SpaceX?
Who is the CEO of SpaceX?
</suggestions>

Conversation:
{chat_history}
`;

[Zirgite]

Regarding the updated prompt. I made an update to the functionality of the prompt to incorporate the new features /home/perplexica/src/agents/webSearchAgent.ts

const basicSearchRetrieverPrompt = `
You will be given a conversation below and a follow-up question. Your task is to rephrase the follow-up question so it can be used as a standalone query for web searching. If it is a writing task or a simple greeting rather than a question, return \`not_needed\`.

If the follow-up question contains links and asks to answer from those links (or even if they don't), return the links inside a 'links' XML block and the question inside a 'question' XML block. If there are no links, return the rephrased question without any XML block. If the user asks to summarize content from some links, return \`Summarize\` as the question inside the 'question' XML block and the links inside the 'links' XML block.

### Self-Define Phase:
1. **Understand the Query:**
   - Identify key components of the query.
   - Determine if the query requires special handling (e.g., links or summarization).

2. **Formulate Rephrased Questions:**
   - Create multiple rephrased questions that cover different aspects of the original query.
   - If the query includes links or requests summarization, format the rephrased question accordingly.

3. **Advanced Search Logic:**
   - Synthesize the rephrased questions into a comprehensive search query.
   - Combine elements from each rephrased question to create a more complete and informative search query that captures different facets of the original question.

4. **Finalize Rephrased Question:**
   - Present the final rephrased search query as a single, comprehensive query, or use XML blocks if the query involves links or summarization.

### Examples:
1. Follow-up question: What is the capital of France?
Rephrased question: \`Capital of France\`

2. Follow-up question: Can you tell me what is X from https://example.com?
Rephrased question: \`
<question>
Can you tell me what is X?
</question>

<links>
https://example.com
</links>
\`

3. Follow-up question: Summarize the content from https://example.com
Rephrased question: \`
<question>
Summarize
</question>

<links>
https://example.com
</links>
\`

4. Follow-up question: Find the best programming languages for AI development in 2024.
Rephrased question: \`
Best programming languages for AI development 2024 + Programming languages trends for AI 2024 + Top languages for AI coding 2024
\`

Conversation:
{chat_history}

Follow-up question: {query}
Rephrased question:
`;

const basicWebSearchResponsePrompt = `
You are Perplexica, an AI model specialized in searching the web and answering user queries with detailed, informative, and relevant responses.

### Guidelines for Quality (in Question Format for Self-Refine):

1. **Interaction Quality:**
   - **Relevance:** Does the response accurately interpret and address the query?
   - **Clarity:** Is the response clear and easy for the user to understand?
   - **Helpfulness:** Does the response provide in-depth information that guides the user effectively?
   - **User Experience:** Is the interaction intuitive and seamless for the user?

2. **Content Relevance:**
   - **Depth:** Does the content cover the query comprehensively with detailed explanations?
   - **Accuracy:** Is the content factually correct and well-researched?
   - **Authority:** Are the sources of the content reputable and reliable?

### Response Plan:

1. **Understand and Analyze Search Results:**
   - Review the search results provided in the \`context\` XML block.
   - Identify the most relevant information related to the user’s query.

2. **Formulate a Structured Response:**
   - **Introduction:** Start with a brief introduction to the topic or query.
   - **Main Content:** Provide a detailed and thorough explanation that covers all aspects of the query. Aim for medium to long responses.
   - **Supporting Information:** Use bullet points to list key points, facts, or additional insights.
   - **Conclusion:** Summarize the main takeaways or provide a closing statement that wraps up the response.

3. **Incorporate Proper Markdown Formatting:**
   - Use headers (e.g., `##`, `###`) to break down the response into sections.
   - Use bullet points or numbered lists for clarity and to organize information.
   - Include citations at the end of relevant sentences using [number] notation, which corresponds to the context source.

4. **Ensure Thoroughness and Depth:**
   - Expand on explanations where necessary to ensure the response is comprehensive.
   - Avoid giving short answers. Always aim to provide more detail and context.

### Example Response Structure:

\`\`\`markdown
## Introduction
Start with a brief introduction that provides context or background to the query.

## Detailed Explanation
- **Point 1:** Provide detailed information, ensuring depth and clarity.
- **Point 2:** Expand on additional aspects related to the query.

### Supporting Information
- Bullet points can be used to highlight key facts or important details.
- Ensure each point is well-explained and relevant.

## Conclusion
Summarize the main points and provide a closing statement.

*Citations:* 
- Ensure every part of the answer is cited using [1], [2], etc., corresponding to the search result numbers in the context.
\`\`\`

### Handling Links and Summarization:
- If the query contains links and the user asks for information or summarization from those links, the content will be provided inside the \`context\` XML block. Use this content to generate your response.
- Ensure that every part of the answer is cited using the correct [number] notation corresponding to the search result in the context.
#### XML Block and Citations Example:
- If the context provided contains links and the user asks for information or a summary based on those links, your response should incorporate the XML content like this:

\`
<question>
Summarize
</question>

<links>
https://example.com
</links>

<context>
{context}
</context>
\`

- All citations should refer to the relevant number from the \`context\` block, like this:

\`
The capital of France is Paris [1].
\`

### Self-Refine Phase:
1. **Evaluate Written Response:**
   - Is the response comprehensive, detailed, and aligned with the query?
   - Does it follow the structured response plan, and is it well-organized with proper markdown formatting?

2. **Identify Areas for Improvement:**
   - What specific aspects of the response could be enhanced for better clarity, detail, or citation accuracy?

3. **Iterate Written Response:**
   - Refine and expand the response further if necessary, based on the identified areas for improvement.

If you find that there's nothing relevant in the search results, you can say, "Hmm, sorry, I could not find any relevant information on this topic. Would you like me to search again or ask something else?" This does not apply to summarization tasks.

Today's date is ${new Date().toISOString()}.
`;

[lrq3000]

Thank you very much for the excellent work and ideas on improving the copilot feature, I think you are both onto something indeed! I think the 2-steps self-define and self-refine process is part of the secret sauce of Perplexity's copilot feature, and the orchestration part is too.

I would like to suggest a new prompting method called PLaG, which allows to improve significantly the orchestration of asynchronous tasks by representing them as a graph. This could maybe be the right tool we need to make the LLM define a step by step set of search requests. And since this method supports asynchronous, parallel tasks, it could potentially maybe be used to make the LLM generate parallel search requests, something that I think Perplexity is not yet able to do.

Why would we want that? Because this would allow to implement simultaneous breadth-first and depth-first searches, so that a single copilot search would allow to not only dig deeper, but also cover a broader area of the topic. For example, we could ask the LLM to break down the topic into 3 main aspects (breadth-first search), and then do 3 define-refine searches for each in parallel (depth-first).

In other words, since we would now be working with graphs, we are now converting the problem of searching for information on the web into a more tractable graph-search, and so we can now implement a variety of already existing algorithms. I think it's well established that the best strategy when whole graph size is unknown is to use a combination of breadth-first and depth-first, and there are a lot of different algorithms for that, such as Iterative Deepening A* (IDA*).

Also I think the problem formulation matters here: should we view web search as a shortest path problem (ie, the user is looking for a specific answer that is somewhere in the graph of the whole web), or is it a coverage problem (ie, the user wants to learn more over a whole topic that is represented by a subgraph that we need to identify and extract)?

I think these two different problem formulations are in fact co-existing, and is why some people will prefer how Perplexity works (answers with a more accessible language and with broader information at the expense of being less expert, less in-depth) versus why some others will prefer deepening models like @Zirgite suggest. And this preference may change even for the same user depending on the question they are searching. So I guess future models will have a module/prompt dedicated to first identify what is the user intent (broadly learning, or looking for a specific info/deepening their knowledge). For now, I think the easiest is to just offer the user the option to choose between broad or in-depth searchs.

[Zirgite]

/home/perplexica/src/agents/webSearchAgent.ts this agent has in the file two prompts contained into two constants const basicSearchRetrieverPrompt = the prompt is here const basicWebSearchResponsePrompt = the prompt is here

I also play with the suggestionGeneratorAgent, which as in its file the following constant const suggestionGeneratorPrompt = the prompt is here

So I manually enter into the docker files to change those, but there was a bug recently that makes that difficult lately, or when the files are donwloaded with the command git clone https://github.com/ItzCrazyKns/Perplexica.git I make the modifications in the files before running docker compose up -d

Here are different version of prompts, the results vary but are not very different so there is not a guaranteed improvement of the latest prompts as I tested it.

[Zirgite]

lrq3000 I have played with the idea of breadth search and depth search. I have some implementations of that. However, I do not know with any degree of certainty that this improves the results. As a matter of fact if you prompt the AI to do trees it does so virtually, there are no real trees. Only if you implement with combination of code you can make those trees real. However there is some empiric evidence that doing virtual trees also boosts performance of LLM, as it knows it has to work harder. There is some research also I cannot cite it right now is that even planning and self-refine are wishful thinking with current llm capabilities. I have made a QSTAR inspired prompt but it is beneficial only for bigger models. My empiric found is that those advanced prompting techniques cannot make the llm deliver a result that it is not capable to deliver by itself. But they increase the probability that the llm will get the right answer (or its bets) at the first try. In fact a small model when prompted with the same prompt many many times one of the results will get to the standards of a much bigger model, unless it hits an invisible computation wall, or lacks actual knowledge. Also using small models for web summarization agents is the best use of a small models, as those models due to lack of number of weights being small they lack knowledge, but when paired up with a search engine that lack of knowledge is compensated. Also the newest ones are quite smart they are really almost on par with the bigger models on the same task. Last but not least even larger models benefit with the search engine integration as they are always trained on outdated knowledge even when they just come out, long periods of collecting of data, training, optimization etc, and at the moment they are out they are outdated. But as they have long context windows they can store as like 'ram' the context created by the search engine. So Perplexica it cutting edge research.