search

VRSEN / agency-swarm-lab

MIT License
522 stars 238 forks source link

Local LLM #8

Open Paritybit01 opened 4 months ago

Paritybit01 commented 4 months ago

Not really an issue per say. Is there any way to get this to use a local LLM server like Ollama or LM Studio instead of having to pay OpenAI fees? Other than that, great work.

phong-phuong commented 4 months ago

It would more of a feature to be added to the wish list (if any).

A layer of abstraction with the LLM layer would be required. Currently each tool related to plan/code generation is directly using the Open AI client which which violates the open/closed principle. However, this would require a rework of the BaseTool which directly uses the OpenAISchema.