Open wishatch opened 11 months ago
Assuming that you're running on corporate Intranet which typically has HTTP proxy setup, then what you're running into is likely the SSL inspection certificate. There's a way to define HTTP proxy for OLLAMA (see https://github.com/jmorganca/ollama/issues/1063) but this is how I solved it:
.dockerignore
with !filename
(where filename
is replaced) to ensure that Docker build pulls the file inDockerfile
and ensure that the certificate (where filename
is replaced) is added to the following places via
ADD filename /etc/ssl/certs/filename
RUN cat /etc/ssl/certs/filename >> /etc/ssl/certs/ca-certificates.crt
RUN cat /etc/ssl/certs/filename >> /etc/local/lib/python-3.11/site-packages/certifi/cacert.pem
Assuming that you're running on corporate Intranet which typically has HTTP proxy setup, then what you're running into is likely the SSL inspection certificate. There's a way to define HTTP proxy for OLLAMA (see jmorganca/ollama#1063) but this is how I solved it:
- You need to download the top-level certificate authority certificate. If you're running Chrome browser when you're on the corporate Intranet, then navigate to an external website, click on the lock icon left of the URL, click on "Connection is secure", click on "Certificate is valid", click on "View Certificate" for the top-level certificate, click on "Details" tab, "Copy to File..." and click "Base-64 encoded X.509 (.CER)"
- Add downloaded certificate to the root directory of this repository after you cloned it
- Add the filename to
.dockerignore
with!filename
(wherefilename
is replaced) to ensure that Docker build pulls the file in- You basically have to tamper with the
Dockerfile
and ensure that the certificate (wherefilename
is replaced) is added to the following places viaADD filename /etc/ssl/certs/filename RUN cat /etc/ssl/certs/filename >> /etc/ssl/certs/ca-certificates.crt RUN cat /etc/ssl/certs/filename >> /etc/local/lib/python-3.11/site-packages/certifi/cacert.pem
@wchan87 Thanks much for your advice. I need a few clarification as I am new to this kind of configuration.
.Dockerfile
because they all seem to connect to some external website in some way.pulling manifest Error: pull model manifest: Get "https://registry.ollama.ai/v2/library/llama3/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority
@wchan87 Same error after applying fixes you have mentioned above
I am using VSCode WSL2, Ubuntu 22.04 and Docker Engine v24.0.6 The .env file contains:
LLM=mistral #or any llama2:7b Ollama model tag, gpt-4, gpt-3.5, or claudev2 EMBEDDING_MODEL=sentence_transformer #or openai, ollama, or aws OLLAMA_BASE_URL=http://llm:11434
Executing this command: docker compose --profile linux up --build Giving me these lines:
Attaching to genai-stack-api-1, genai-stack-bot-1, genai-stack-database-1, genai-stack-front-end-1, genai-stack-llm-1, genai-stack-loader-1, genai-stack-pdf_bot-1, genai-stack-pull-model-1 genai-stack-pull-model-1 | pulling ollama model mistral using http://llm:11434 genai-stack-llm-1 | [GIN] 2023/11/02 - 10:47:24 | 200 | 57.058µs | 172.18.0.2 | HEAD "/" genai-stack-pull-model-1 | pulling manifest genai-stack-llm-1 | 2023/11/02 10:47:26 images.go:1164: couldn't get manifest: Get "https://registry.ollama.ai/v2/library/mistral/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority genai-stack-llm-1 | [GIN] 2023/11/02 - 10:47:26 | 200 | 1.672986504s | 172.18.0.2 | POST "/api/pull" genai-stack-pull-model-1 | Error: pull model manifest: Get "https://registry.ollama.ai/v2/library/mistral/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority service "pull-model" didn't complete successfully: exit 1
Please advice on how to fix the error. Thx much.