search

CsabaConsulting / ThruThinkCohereWeaviateChat

Cohere and Weaviate powered ThruThink support chat on Streamlit
https://thruthinksupport.streamlit.app/
MIT License
1 stars 3 forks source link

Executing the variation document retrievals and the twin co.chat queries paralelly #4

Closed MrCsabaToth closed 9 months ago

MrCsabaToth commented 9 months ago

Use asyncio in Streamlit, we can parallelize:

  1. The document retrieval for the variations
  2. The twin calls (document based and web connector) of co.chat

(The latter two are the real latency hogs)

MrCsabaToth commented 9 months ago

Trying asyncio along the lines of https://discuss.streamlit.io/t/streamlit-and-asynchronous-functions/30684 failed. The UI freezes, doesn't initialize

MrCsabaToth commented 9 months ago

Let's try threading? https://discuss.streamlit.io/t/concurrency-with-streamlit/29500/7