Quansight / ragna

RAG orchestration framework ⛵️
https://ragna.chat
BSD 3-Clause "New" or "Revised" License
179 stars 22 forks source link

remove max_input_size property from assistants #362

Closed pmeier closed 8 months ago

pmeier commented 8 months ago

The abstract Assistant.max_input_size property was a left over from the prototype implementation we started from. The idea behind it was for to be able to automatically fill the context window, such that it is maxed out with the sources and the user prompt.

However, this makes quite a few assumptions, that held for our prototype, but are no longer true for Ragna:

Meaning, Assistant.max_input_size is not only unused, actually using it requires a major effort in designing a proper interface. And with all that uncertainty I'd rather not force users to implement this property.

Note that this is only mildly BC breaking. Users that had defined this property on their custom assistants are untouched by this change. It just stays unused. Only if you access this property on our builtin assistants, you will have a breakage.

pmeier commented 8 months ago

Cc @smokestacklightnin. This should also make #358 easier.