Stream chat completion in Streamlit apps

[slevin48]

https://medium.com/@avra42/how-to-stream-output-in-chatgpt-style-while-using-openai-completion-method-b90331c15e85 https://www.youtube.com/watch?v=CqqELxWGUy8 https://github.com/avrabyt/OpenAI-Streamlit-YouTube/tree/451de7a5b9b2cfcd55ff828e0ddb213f2274cf8e

import openai
import streamlit as st
from streamlit_pills import pills

openai.api_key = st.secrets['api_secret']

st.subheader("AI Assistant : Streamlit + OpenAI: `stream` *argument*")

# You can also use radio buttons instead
selected = pills("", ["NO Streaming", "Streaming"], ["🎈", "🌈"])

user_input = st.text_input("You: ",placeholder = "Ask me anything ...", key="input")

if st.button("Submit", type="primary"):
    st.markdown("----")
    res_box = st.empty()

    if selected == "Streaming":
        report = []
        # Looping over the response
        for resp in openai.Completion.create(model='text-davinci-003',
                                            prompt=user_input,
                                            max_tokens=120, 
                                            temperature = 0.5,
                                            stream = True):
            # join method to concatenate the elements of the list 
            # into a single string, 
            # then strip out any empty strings
            report.append(resp.choices[0].text)
            result = "".join(report).strip()
            result = result.replace("\n", "")        
            res_box.markdown(f'*{result}*') 

    else:
        completions = openai.Completion.create(model='text-davinci-003',
                                            prompt=user_input,
                                            max_tokens=120, 
                                            temperature = 0.5,
                                            stream = False)
        result = completions.choices[0].text

        res_box.write(result)
st.markdown("----")

[slevin48]

Change model to gpt-3.5-turbo

[slevin48]

https://docs.streamlit.io/library/api-reference/chat https://docs.streamlit.io/knowledge-base/tutorials/build-conversational-apps

[slevin48]

https://blog.streamlit.io/langchain-streamlit/ https://github.com/langchain-ai/streamlit-agent https://github.com/langchain-ai/streamlit-agent/blob/main/streamlit_agent/basic_streaming.py

[slevin48]

streamlit-streamlit_app-2023-07-15-22-07-70.webm