Open iChristGit opened 9 months ago
you can. in the .env just change the base url api
u have to launch text-generation with the openai extention. the base url then becomes https://127.0.0.1:5000
Tools Select what external tools your agents have access to.
Error loading tools.
Language įŽäŊä¸æ
API Key You can optionally use your own API key here. You can find your API key in your OpenAI dashboard.
Test Advanced Settings Model gpt-3.5-turbo
and http://localhost:8051/v1 is local llm api,but it not work.
Ehhh, textgen webui does not support function calling at this moment, so the api call would fail immediately
we getting closer for ooba's function calling support: PR 5185 is nearly ready to be merged.
cd next/src grep -R "openai"
there is openai.com hardcoded and not using .env
server/api/routers/agentRouter.ts the functionnew OpenAI() has no baseURL:
Harcoded OpenAPI url should be replace by configurable so we can use any custom locally running LLM.
â ī¸ Please check that this feature request hasn't been suggested before.
đ Feature description
This is the project: https://github.com/oobabooga/text-generation-webui/ It has by default an openai like api and it all runs locally, can it be support with this project?
âī¸ Solution
Add support for TextGenWebui
â Alternatives
None
đ Additional Context
No response
Acknowledgements