Closed johnaiii closed 2 months ago
Getting the same issue running lm-studio, and fabric through WSL.
I got models to list by manually adding the api key to the .env config
I came across the same issue. There was an ollama instance running in WSL, and I couldn't figure out why it wasn't sending prompts to LM-Studio.
@danielmiessler - I see you are holding most updates for after the GO version is released, not sure how that will change the setup.
From what I can tell having the API_KEY set is required by the OpenAI python package.
# Making sure the paths are set up correctly
fabric --help
[!NOTE] If you're using the
server
functions,fabric-api
andfabric-webui
need to be run in distinct terminal windows.
If you want to use fabric with OpenAI API compatible inference servers, such as FastChat, Helmholtz Blablador, LM Studio and others, simply export the following environment variables:
export OPENAI_BASE_URL=https://YOUR-SERVER:8000/v1/
export DEFAULT_MODEL="YOUR_MODEL"
And if your server needs authentication tokens, like Blablador does, you export the token the same way you would with OpenAI:
export OPENAI_API_KEY="YOUR TOKEN"
If using LM-Studio you must set your API key to lm-studio
export OPENAI_API_KEY="lm-studio"
[!NOTE] When running in WSL2 your fabric CLI client must be pointed at your computers network IP address, not https://localhost or https://127.0.0.1 For example
export OPENAI_BASE_URL=https://192.168.0.10:1234/v1/
fabric
clientOnce you have it all set up, here's how to use it.
fabric -h
Thanks! I could never get this working.
On Sat, Jun 1, 2024 at 9:03 PM Sean Visnaw @.***> wrote:
I came across the same issue. There was an ollama instance running in WSL, and I couldn't figure out why it wasn't sending prompts to LM-Studio.
@danielmiessler https://github.com/danielmiessler - I see you are holding most updates for after the GO version is released, not sure how that will change the setup.
From what I can tell having the API_KEY set is required by the OpenAI python package. Below is the proposed update to the documentation.
- Now you are up and running! You can test by running the help.
Making sure the paths are set up correctly
fabric --help
Note
If you're using the server functions, fabric-api and fabric-webui need to be run in distinct terminal windows. Using a local OpenAI API compatible server
If you want to use fabric with OpenAI API compatible inference servers, such as FastChat https://github.com/lm-sys/FastChat, Helmholtz Blablador http://helmholtz-blablador.fz-juelich.de, LM Studio https://lmstudio.ai and others, simply export the following environment variables:
- export OPENAI_BASE_URL=https://YOUR-SERVER:8000/v1/
- export DEFAULT_MODEL="YOUR_MODEL"
And if your server needs authentication tokens, like Blablador does, you export the token the same way you would with OpenAI:
- export OPENAI_API_KEY="YOUR TOKEN"
If using LM-Studio you must set your API key to lm-studio
- export OPENAI_API_KEY="lm-studio"
Note
When running in WSL2 your fabric CLI client must be pointed at your computers network IP address, not https://localhost or https://127.0.0.1 For example export OPENAI_BASE_URL=https://192.168.0.10:1234/v1/ Using the fabric client
Once you have it all set up, here's how to use it.
- Check out the options fabric -h
— Reply to this email directly, view it on GitHub https://github.com/danielmiessler/fabric/issues/489#issuecomment-2143652912, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPVKQWVTVEUKJF3GBHJCS3ZFJVPNAVCNFSM6AAAAABIUALZGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGY2TEOJRGI . You are receiving this because you are subscribed to this thread.Message ID: @.***>
-- Neil M. Greisman @.*** 416-277-8907
Same problem here with openai models.
fabric --listmodels
gives me just an empty result. Trying to summarize something with
fabric --text "test" --pattern summarize
results in:
Error: Connection error.
Connection error.
Environment: Windows 10 with WSL2 + Ubuntu 22.04.4 LTS ~/.config/fabric/.env exists + API key is configured.
cat ~/.config/fabric/.env
YOUTUBE_API_KEY=XXXXXXXX
OPENAI_API_KEY=XXXXXXXX
I also tried setting the api key with "export OPENAI_API_KEY="YOUR TOKEN"" and even "export OPENAI_BASE_URL="https://api.openai.com/v1""
What happened?
I was installing fabric, when I wrote: ":~/fabric$ fabric --listmodels" and local models are empty, when I am running Ollama in Windows 11: :~/fabric$ fabric --listmodels GPT Models:
Local Models:
Claude Models:
Google Models:
Version check
Relevant log output
No response
Relevant screenshots (optional)
No response