alan-turing-institute / reginald

Reginald repository for REG Hack Week 23
3 stars 0 forks source link

Add Llama-2 chat engine model to slack #70

Closed rchan26 closed 12 months ago

rchan26 commented 1 year ago

From #64 and #65, we have been able to replace the LLM with a quantized version llama-2 which uses far less memory than the standard and still seems to provide good performance. Further, we have looked at implementing a chat engine variant (rather than a query engine) of the model. Lets add this model to slack.

rchan26 commented 1 year ago

Need to consider whether or not this constitutes a new class, or we can add functionality within LlamaIndex to specify whether a chat engine is preferred over a query. It feels like the latter would be nicer to use, but I wonder if it might get messy...

rchan26 commented 1 year ago

For the multiple chat instances. When we receive a message from Slack, we do also get their user id, so this might be useful