search

CsabaConsulting / InspectorGadgetApp

Open Multi-Modal Personal Assistant
MIT License
4 stars 1 forks source link

Employing Small-to-Big Retrieval #58

Open MrCsabaToth opened 6 hours ago

MrCsabaToth commented 6 hours ago

After matching specific entries in history, we should expand the stuffed blocks with some more entries along the matched entry. For example if the entry contained multiple function call turns or additional media attachments go through that and include a few more steps back and forward in the history.

MrCsabaToth commented 5 hours ago

Since in our case it's not about chunks, we could go with this simple heuristic first:

  1. If the retrieval matches on a user utterance, we should look further in history (skipping any medium entries and optionally include any function call entries) and add at least the model response to that user input.
  2. If the retrieval matches on a model response, we should look back in history (skipping any medium entries and optionally include any function call entries) and add at least the user request triggering the model response.

Maybe later we can think about adding even more rounds.