search

josephrocca / OpenCharacters

Simple little web interface for creating characters and chatting with them. It's basically a single HTML file - no server. Share characters using a link (character data is stored within the URL itself). All chat data is stored in your browser using IndexedDB. Currently supports OpenAI APIs and ~any Hugging Face model.
https://josephrocca.github.io/OpenCharacters
MIT License
364 stars 60 forks source link

Optimization suggestion: Reuse previous memory/lore queries when generating variant responses #57

Open kickahaota opened 1 year ago

kickahaota commented 1 year ago

When the user clicks the blue double-arrow button to generate a new variant response, it looks like the memory and lore queries are repeated. That seems a bit inefficient and expensive. Can the previous memory/lore results be reused instead?

josephrocca commented 1 year ago

Not sure that this makes sense in all/most cases - the user may be retrying because (whether they know or not) the queries were bad. The memory queries are very cheap anyway (compared to bot responses - EDIT: actually this may be wrong, can't remember how much context memory-query-generations use. they should be cheaper - but not sure by how much), so it'd only be worth it to save the user some time (but again, not sure it would make sense in all/most cases due to aforementioned reason)