Local Inferencing Endpoint Only? Still references api.aponeai.com/v1/models

[xJohnWhite]

Discussed in https://github.com/danielmiessler/fabric/discussions/544

^{Originally posted by **xJohnWhite** June 4, 2024} # Summary Using an inside-my-network OpenAI API-compatible inferencing endpoint with relevant env set, but getting an authentication error from api.openai.com # Details I'm at an organization that doesn't allow the use of OpenAI; Fortunately, I have access to an internal inferencing endpoint which is OpenAI API-compatible. ## Env check 1 I've got my internal api set up in my environment variables: ```shell printenv | grep OPENAI OPENAI_MODEL_NAME=llama27b OPENAI_API_KEY=[myInternalKey] OPENAI_API_BASE=https://[myEndpointDomain]/api/v1/ ``` ## Env check 2 I've got that in the env as well. ```shell grep OPENAI ~/.config/fabric/.env OPENAI_API_BASE="https://[myEndpointDomain]/api/v1/" OPENAI_MODEL_NAME="llama27b" OPENAI_API_KEY="[myInternalKey]" ``` ## OpenAPI.com Authentication error However, when I try to do anything, I get an OpenAI URL error, as if it's hard-coded somewhere: ```shell echo "Summer is the best season" | fabric -p write_micro_essay /home/jwhite/.local/pipx/venvs/fabric/lib/python3.10/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning) Error: Client error '401 Unauthorized' for url 'https://api.openai.com/v1/models' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401 ``` Am I doing something massively wrong? Oh, and what's with the ffmpeg/avconv error? ## Previously mentioned #189 Saw a previously merged #189 , but I'm using a git clone from today and checked installer/client/cli/utils.py to make sure the merge was still there. There's something in def agents(self, userInput): that has a hard-coded check to see if the model name is in self.model, but I'm hoping that I'm not invoking that without a specific agent creation pattern. ## Potential other installer/client/cli/utils.py OpenAI hard-code In installer/client/cli/utils.py, lines 43-45, there's this: ```python if "OPENAI_API_KEY" in os.environ: api_key = os.environ['OPENAI_API_KEY'] self.client = OpenAI(api_key=api_key)self.client = OpenAI(api_key=api_key) ``` I'm not familiar enough with the OpenAI invocation to know whether it picks up OPENAI_BASE_URL and OPENAI_API_MODEL from the environment. I've seen other people referencing using non-localhost, non-Ollama servers, so I'm guessing I've got a misconfiguration somewhere. ## Request for guidance and wisdom! Any thoughts on what might be going on?

[xJohnWhite]

No attention in Q&A, so created an issue.

[bamit99]

Have you tried doing it over clear http rather than ssl. Try removing the certificate for testing purposes. That being said, I had better luck with LMStudio rather than Ollama. Seems your setup is on the same lines

[mmartoccia]

Related, new install with appropriate keys produces this same 401 error.

[bamit99]

Read #405