Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.
9.95k
stars
1.4k
forks
source link
[OctoAI] Llama-3 based summarization + RAG to power a sales bot #500
What does this PR do?
Introducing a new Llama3 example that is built on top of OctoAI model API provider and Weaviate vector DB to implement a helpful sales bot.
Example interaction:
User: "what is must have accessory for my new electric guitar"
Chatbot:
Feature/Issue validation/testing
Ran on a local jupyter notebook on Macbook M1.
Before submitting
Thanks for contributing 🎉!