search

vanna-ai / vanna

🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using RAG 🔄.
https://vanna.ai/docs/
MIT License
11.04k stars 862 forks source link

vllm support added #386

Closed Navanit-git closed 5 months ago

Navanit-git commented 5 months ago

Hi, so for open llm. I was trying to work something. Using vLLM, I was able to do the query using llama 3 and other models in my gpu. I hope it will be a good asset. Kindly review and tell me any changes that is required.

zainhoda commented 5 months ago

Thanks! This looks pretty good. I'll have to try it to verify before merging

Navanit-git commented 5 months ago

Sure. I have tried using on some sample code having sqlite and queries. Do tell me if you require something else