search

alan-turing-institute / reginald

Reginald repository for REG Hack Week 23
3 stars 0 forks source link

Try out Mixtral-8x7B model #149

Closed rchan26 closed 2 months ago

rchan26 commented 8 months ago

Mistral's Mixtral 8x7b model is a new open-source model which is competitive with GPT3.5: https://mistral.ai/news/mixtral-of-experts/

There are now quantised versions of this model available on Huggingface which we can experiment with and using with our implementation of llama-index-llama-cpp.

Alternatively, we can try out Mistral's Mistral 7B model (with llama-index-hf) or quantised versions of it.

Feel free to experiment with Mistral's models in a notebook setting like here or with the full pipeline.

rchan26 commented 2 months ago

Closing since we can easily try out different VMs using the Ollama model implemented in #179