search

theopenconversationkit / tock

Tock, the open source conversational AI toolkit.
https://doc.tock.ai
Apache License 2.0
504 stars 132 forks source link

1668 spike flashrerank reranking avec petit modèle sur cpu #1687

Closed morgandiverrez closed 5 days ago

morgandiverrez commented 4 months ago

dercbot 1047

This Spike won't be merge, it's for experimental purposes we are checking if providing reranking with small models (running on CPU) could provide good result an evaluating the RAM usage cost of this method.

For this experimentation, we target only the Gen AI Orchestrator, we will not alter the Bot API so that we can test it by only deploying the Gen AI Orchestrator with configurations based on environment variables.

Benvii commented 5 days ago

This is replaced by https://github.com/theopenconversationkit/tock/pull/1721 I'm closing it as it as too much impact on the size of the orchestrator image.