Local LLM support - Githubissues

mishushakov / llm-scraper

Turn any webpage into structured data using LLMs

MIT License

2.42k stars 147 forks source link

Local LLM support #1

Closed mishushakov closed 7 months ago

mishushakov commented 7 months ago

We want to also support local LLMs with functions calling

mintisan commented 7 months ago

like Ollama API wit base url

mishushakov commented 7 months ago

The problem is that Ollama currently doesn't support function calling and grammars. Until then, I'm working on a solution based on node-llama-cpp, which does in fact support it

mishushakov commented 7 months ago

have something working already

mishushakov commented 7 months ago

This is now supported (see readme)