how to run with local LLM

whyhow-ai / knowledge-table

Knowledge Table is an open-source package designed to simplify extracting and exploring structured data from unstructured documents.

MIT License

344 stars 47 forks source link

how to run with local LLM #67

Open viegoxu01 opened 6 days ago

viegoxu01 commented 6 days ago

I have run a vllm proxy server with my fine-tuned local llm, and have the URL for the vLLM proxy server. How can I use it within Knowleadge-Table in the same way as OpenAI servers. THX

daboe01 commented 4 days ago

ollama-support would be fine.