Add Llama-2 model to existing available query engines - Githubissues

alan-turing-institute / reginald

Reginald repository for REG Hack Week 23

3 stars 0 forks source link

Add Llama-2 model to existing available query engines #78

Closed rchan26 closed 1 year ago

rchan26 commented 1 year ago

We only have query engines at the moment, so adding Llama-2 is a slightly easier task than #70.

[x] Add LlamaIndexLlamaCPP (or similar name) class which is a subclass of LlamaIndex
[x] Make the model path (to the GGUF file) something we pass in to this class (see how we pass in model name into LlamaIndexHF)
[x] Make _prep_llm method which constructs the LLM from llama_index.LlamaCPP
[x] Maybe make n_gpu_layers a parameter to pass in too?

Perhaps better to start with this before #70.