search

C-Loftus / talon-ai-tools

Query LLMs and AI tools with voice commands
http://colton.place/talon-ai-tools/
MIT License
54 stars 21 forks source link

First pass at tracking thread roles #142

Closed jaresty closed 2 months ago

jaresty commented 2 months ago
jaresty commented 2 months ago

I'm going to use this for a while, but I think it's working now.

jaresty commented 2 months ago

I think this is working pretty well now. Let me know if you have any feedback or if you'd like me to merge it-I think it would be good to get this in before any other big changes.

jaresty commented 2 months ago

Importantly this introduces a difference in the way that we are storing threads versus context:

  1. context continues to be stored as an array of messages
  2. threads are now stored as an array of messages at one layer up including the roles

This is because context is now always part of the system role, while threads vary from assistant and user. A future refactor might look at making the context stored in the same way however we build up system context with every request so it might be fine just as is. Tracking whether the messages are coming from the assistant, the user, or in the system context more accurately seems to have a subtle yet good impact on the responses I'm getting.

C-Loftus commented 2 months ago

Generally looks good to me with a comment about the naming of fns. May make some clean up edits in a new PR later today after this is merged.