Open raychz opened 2 months ago
@raychz thanks for the detailed suggestion!
The way I was thinking about this feature is as follows:
Should there be a limit on the number of knowledge bases that can be selected? I don't think there should be.
Yeah, agreed. Wouldn't set a limit.
How might this affect the model's context window
This is an important detail, adding a bunch of context at once could easily exceed the context window of the model.
This is where allowing the user to edit the Ollama
num_ctx
param would come in handy.
In general, I'd like to avoid expecting the user to know which value to set num_ctx
.
It'd be great if we can count the tokens of the current context + prompt (#7) before submitting the form. Looks like there is an ongoing discussion about it: https://github.com/ollama/ollama/issues/1716
If Ollama doesn't have a tokenizer we can use, can we use a 3rd party one?
The other caveat is that each model has a different context window, does Ollama report this value via the API?
Here's where I'm at:
I might have time to take this one on soon if you'd like, but I know you're working on a major UI overhaul, so maybe I'll wait until that one is merged in.
The UI overhaul should have an impact on some of these features but should be minimal for tasks 1 and 4.
[*] I actually have a prototype Chrome extension that extracts the text from the current tab, should make that repo public here's the repo:
Here's a rough mockup for adding multiple text boxes in a particular "Knowledge":
This feature will likely be covered by #116
Support for Multiple Knowledge Bases per Session
Current Situation
Currently, Hollama allows users to select a single knowledge entry per session. This limits the flexibility and depth of information that can be incorporated into a single conversation.
Feature Request
We propose adding support for selecting and using multiple knowledges within a single session. This would allow users to combine different sources of information, creating more comprehensive and versatile conversational contexts.
Proposed Functionality
User Interface Updates:
Logic Changes:
Benefits
Implementation Considerations
num_ctx
param would come in handy. See Ollama docs.Does Ollama support multiple
CONTEXT
tags? Here's a relevant snippet fromollama.ts
.Message
interface would need to be updated to support a list ofKnowledges
. Maybe these two should be decoupled? Or better yet, maybe only theSession
interface should contain the list ofKnowledge
's. Here's the relevant snippet fromsessions.ts
:export interface Session { id: string; model: string; messages: Message[]; context: number[]; knowledge?: Knowledge; // Make this an array of Knowledges? }