search

Stability-AI / StableLM

StableLM: Stability AI Language Models
Apache License 2.0
15.85k stars 1.04k forks source link

What's the proper way to implement chatting feature? #67

Closed finom closed 1 year ago

finom commented 1 year ago

Hey guys! To run the basic example I can use labels like "<|SYSTEM|>", "<|USER|>", "<|ASSISTANT|>". That's clear how to run the example script and how to ask one question but I don't really understand how can I turn that into a chat so the LM has understanding in what has been discussed before. As far as I understand I need to concat all the conversation like 

"""
<|SYSTEM|>...
<|USER|>How are you?
<|ASSISTANT|>Good
<|USER|>In other words?
<|ASSISTANT|>Amazing
"""

and pass it to tokenizer as a prompt. When a new message appears I just append it to the end of the prompt. Is that correct?

RomboutKarelse commented 1 year ago

Did you see this? https://github.com/Stability-AI/StableLM/blob/main/notebooks/stablelm-alpha.ipynb

mcmonkey4eva commented 1 year ago

That looks about correct yes. Make sure to also set stop strings to cut the LLM off if it starts to generate any of the special tokens (ie so it doesn't generate a user reply and start talking to itself)

finom commented 1 year ago

@RomboutKarelse @mcmonkey4eva as far as I understand I need to send the entire chat history as I'd do with OpenAI API?

mcmonkey4eva commented 1 year ago

Yes correct. The LLM doesn't have any internal memory beyond what you feed into it.

RomboutKarelse commented 1 year ago

Ah, I should have pointed you to this code. https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat/blob/main/app.py This implements a chatbot with history.

finom commented 1 year ago

@RomboutKarelse @mcmonkey4eva thank you guys!