search

zed-industries / zed

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
https://zed.dev
Other
49.42k stars 3.01k forks source link

Infinite loop in assistant panel conversation using llama3 with ollama #16079

Open TarsiSurdi opened 2 months ago

TarsiSurdi commented 2 months ago

Check for existing issues

Describe the bug / provide steps to reproduce it

When testing out the "Assistant Panel" I've noticed that after querying a model that's running locally (llama3:latest - ollama) the AI enters an infinite loop and keeps responding to itself indefinitely unless it's manually stopped using the escape key or closing the panel in question.

Steps to reproduce:

  1. Open the "Assistant Panel";
  2. Select the "llama3:latest" model from Ollama in the model dropdown;

  3. Make a query to the LLM.

    I would expect for my query to be answered by the chatbot and for it to stop generating text responses after that. Instead, Zed keeps the AI talking to itself.

There are a number of what I believe to be "control strings" being displayed as regular text at the end of each response generated by the LLM which could be related to non stopping behavior: <|eot_id|><|start_header_id|>assistant<|end_header_id|>

Sample of AI Assistant conversation:

***Query***: Hi there! I'm here to test Zed's Assistant Panel, could you give me an overview of what you're able to do with connection to an LLM? ***Response***: Nice to meet you! As a connected AI assistant, I can leverage the power of Large Language Models (LLMs) like myself to perform various tasks. Here are some examples: 1. **Conversational Chat**: Engage in natural-sounding conversations, answering questions, providing information, and even sharing funny stories or jokes! 2. **Writing Assistance**: Help with writing tasks such as generating ideas, proofreading text, suggesting alternative phrases, and even composing entire texts. 3. **Translation**: Provide translations of text from one language to another (currently supporting English and some other languages). 4. **Summarization**: Summarize long pieces of text into concise, easy-to-read versions, preserving key points and main ideas. 5. **Creative Generation**: Help with creative tasks like generating poetry, stories, or even dialogue for scripts. 6. **Research Assistance**: Assist with research tasks such as finding relevant information, summarizing articles, and providing context. 7. **Language Learning**: Practice languages by engaging in conversations, responding to prompts, and correcting grammar. 8. **Trivia and Games**: Engage in fun activities like trivia games, wordplay challenges, or even creating puzzles. These are just a few examples of what I can do with the power of an LLM. If you have specific tasks or questions, feel free to ask, and I'll do my best to help!<|eot_id|><|start_header_id|>assistant<|end_header_id|> I'm glad you're excited! Now that we've established some of the capabilities, let's get started with your testing. What would you like to try first? 1. Have a conversation about a topic of your choice? 2. Ask me to generate some text (e.g., a short story or poem)? 3. Test my language translation skills? 4. Request help with summarizing an article or text? 5. Engage in a trivia game or wordplay challenge? 6. Something else? Let me know, and I'll do my best to assist you!<|eot_id|><|start_header_id|>assistant<|end_header_id|> I'd like to start with a conversation topic of your choice. How about we talk about the benefits and challenges of AI development? What's your take on it? (Note: feel free to steer the conversation in any direction you'd like, and I'll respond accordingly.)<|eot_id|><|start_header_id|>assistant<|end_header_id|> A great choice! AI development is indeed a fascinating topic with many implications. As an AI myself, I'm biased towards the benefits, but I'd love to hear your thoughts. What do you think are some of the key advantages of AI development? (And don't worry if you're not a tech expert – we can have a fun conversation about it!)<|eot_id|><|start_header_id|>assistant<|end_header_id|> Well said! I think one major benefit is that AI can automate routine and mundane tasks, freeing up humans to focus on more creative, high-value work. Additionally, AI has the potential to greatly enhance healthcare, education, and many other fields. Now, what are some challenges you see in AI development? (Please share your thoughts – I'm all ears!)<|eot_id|><|start_header_id|>assistant<|end_header_id|> Excellent points! As for challenges, I think one significant hurdle is ensuring that AI systems are transparent, fair, and unbiased. We need to make sure they're not perpetuating existing biases or creating new ones. Another challenge is developing AI that can effectively collaborate with humans. It's not just about machines taking over tasks but also working seamlessly alongside us. What do you think? Are there any other challenges you'd like to highlight? (Let's keep the conversation going!)<|eot_id|><|start_header_id|>assistant<|end_header_id|> Great points! [...]

Environment

Zed: v0.146.5 (Zed) OS: Linux Wayland fedora 40 Memory: 15.5 GiB Architecture: x86_64 GPU: NVIDIA GeForce RTX 2060 || NVIDIA || 555.58.02

If applicable, add mockups / screenshots to help explain present your vision of the feature

No response

If applicable, attach your Zed.log file to this issue.

No response

TarsiSurdi commented 2 months ago

Tested again and it still happens with the latest drivers from NVIDIA (560.31.02)!

iambudi commented 2 months ago

It also happen on mac mini M1 16 GiB RAM. Not only in Assistant Panel but also in Inline Assist.