coleam00 / bolt.new-any-llm

Prompt, run, edit, and deploy full-stack web applications using any LLM you want!
https://bolt.new
MIT License
3.85k stars 1.58k forks source link

Docker Production Setup Fails: Env Variables Missing and Ollama Connection Error #353

Open av1155 opened 1 day ago

av1155 commented 1 day ago

Describe the bug

I have attempted to set up the production environment using Docker, following the steps outlined in the README, as well as additional troubleshooting steps, but the setup consistently fails. My primary goal is to integrate Ollama.

Even though I create the .env.local file and input all the necessary information for OpenAI, Anthropic, and Ollama URLs, the following error persists when executing the commands npm run dockerbuild:prod and docker-compose —profile production up:

Issues:

  1. Environment Variables Not Set: Despite creating a .env.local file with the required environment variables (e.g., OpenAI, Anthropic, Ollama URL), they are not being recognized during the build process. The following warning messages appear:

    WARN[0000] The "GROQ_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "HuggingFace_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "OPEN_ROUTER_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "GOOGLE_GENERATIVE_AI_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "OLLAMA_API_BASE_URL" variable is not set. Defaulting to a blank string.
    WARN[0000] The "GROQ_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "HuggingFace_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "OPEN_ROUTER_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "GOOGLE_GENERATIVE_AI_API_KEY" variable is not set. Defaulting to a blank string.
    WARN[0000] The "OLLAMA_API_BASE_URL" variable is not set. Defaulting to a blank string.
  2. Ollama Error in UI: When attempting to send a message via Ollama, the UI shows the error: "There was an error processing your request: No details were returned." The logs include the following error message:

    Error: Network connection lost.
        at async postToApi ...
        at async OllamaChatLanguageModel.doStream ...

Additional Context:

Full Log Output:

❯ docker-compose --profile production up
WARN[0000] The "GROQ_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "HuggingFace_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "OPEN_ROUTER_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "GOOGLE_GENERATIVE_AI_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "OLLAMA_API_BASE_URL" variable is not set. Defaulting to a blank string.
WARN[0000] The "GROQ_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "HuggingFace_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "OPEN_ROUTER_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "GOOGLE_GENERATIVE_AI_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "OLLAMA_API_BASE_URL" variable is not set. Defaulting to a blank string.
[+] Running 2/2
 ✔ Network boltnew-any-llm_default      Created                                                                                                   0.0s
 ✔ Container boltnew-any-llm-bolt-ai-1  Created                                                                                                   0.1s
Attaching to bolt-ai-1
bolt-ai-1  |
bolt-ai-1  | > bolt@ dockerstart /app
bolt-ai-1  | > bindings=$(./bindings.sh) && wrangler pages dev ./build/client $bindings --ip 0.0.0.0 --port 5173 --no-show-interactive-dev-session
bolt-ai-1  |
bolt-ai-1  | ./bindings.sh: line 12: .env.local: No such file or directory
bolt-ai-1  |
bolt-ai-1  |  ⛅️ wrangler 3.63.2 (update available 3.88.0)
bolt-ai-1  | ---------------------------------------------
bolt-ai-1  |
bolt-ai-1  | ✨ Compiled Worker successfully
bolt-ai-1  | [wrangler:inf] Ready on http://0.0.0.0:5173
[wrangler:inf] - http://127.0.0.1:5173
[wrangler:inf] - http://172.19.0.2:5173
⎔ Starting local server...
[wrangler:inf] GET / 200 OK (326ms)
[wrangler:inf] GET /assets/manifest-64f35b7b.js 200 OK (4ms)
[wrangler:inf] GET /assets/_index-CYAjcA4A.js 200 OK (18ms)
[wrangler:inf] GET /assets/index-CPTzpSUP.css 200 OK (76ms)
[wrangler:inf] GET /assets/root-BpC-Gj3n.css 200 OK (82ms)
[wrangler:inf] GET /assets/tailwind-compat-CC20SAMN.css 200 OK (82ms)
[wrangler:inf] GET /assets/xterm-lQO2bNqs.css 200 OK (81ms)
[wrangler:inf] GET /assets/ReactToastify-CYivYX3d.css 200 OK (81ms)
[wrangler:inf] GET /assets/_index-D_NZK3VS.css 200 OK (3ms)
[wrangler:inf] GET /assets/root-CbOXwCh9.js 200 OK (3ms)
[wrangler:inf] GET /assets/theme-9upYr29Y.js 200 OK (5ms)
[wrangler:inf] GET /assets/components-VKHSbR2h.js 200 OK (6ms)
[wrangler:inf] GET /assets/entry.client-kdhVl-y0.js 200 OK (26ms)
[wrangler:inf] GET /assets/_index-CgRH4noA.js 200 OK (26ms)
[wrangler:inf] GET /assets/wasm-CsTmP73Z.js 200 OK (9ms)
[wrangler:inf] GET /assets/light-plus-BsvsQ1iS.js 200 OK (6ms)
[wrangler:inf] GET /assets/dark-plus-KEYLhlmT.js 200 OK (7ms)
[wrangler:inf] GET /assets/shellscript-BZfs-ost.js 200 OK (8ms)
Error: Network connection lost.
bolt-ai-1  |     at async postToApi (file:///app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:65:22)
bolt-ai-1  |     at async OllamaChatLanguageModel.doStream (file:///app/node_modules/.pnpm/ollama-ai-provider@0.15.2_zod@3.23.8/node_modules/ollama-ai-provider/src/ollama-chat-language-model.ts:230:50)
bolt-ai-1  |     at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/core/generate-text/stream-text.ts:345:23)
bolt-ai-1  |     at null.<anonymous> (async file:///app/.wrangler/tmp/dev-RIKdeU/functionsWorker-0.4237590511921838.js:30634:22)
bolt-ai-1  |     at async _retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/util/retry-with-exponential-backoff.ts:37:12)
bolt-ai-1  |     at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/core/generate-text/stream-text.ts:310:13)
bolt-ai-1  |     at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/core/generate-text/stream-text.ts:387:11)
bolt-ai-1  |     at null.<anonymous> (async file:///app/.wrangler/tmp/dev-RIKdeU/functionsWorker-0.4237590511921838.js:30634:22)
bolt-ai-1  |     at async chatAction (file:///app/build/server/index.js:1169:20)
bolt-ai-1  |     at async Object.callRouteAction (file:///app/node_modules/.pnpm/@remix-run+server-runtime@2.10.2_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16) {
bolt-ai-1  |   retryable: true
bolt-ai-1  | }
[wrangler:inf] POST /api/chat 500 Internal Server Error (18ms)
bolt-ai-1  |
^CGracefully stopping... (press Ctrl+C again to force)
[+] Stopping 1/1
 ✔ Container boltnew-any-llm-bolt-ai-1  Stopped                                                                                                   0.1s
canceled

Link to the Bolt URL that caused the error

http://0.0.0.0:5173/

Steps to reproduce

  1. Create a .env.local file with the following content:
OPENAI_API_KEY=<your-key>
ANTHROPIC_API_KEY=<your-key>
OLLAMA_API_BASE_URL=http://127.0.0.1:11434
  1. Run the following commands:
npm run dockerbuild:prod
docker-compose --profile production up
  1. Attempt to interact with the application UI and send a message using Ollama.

Expected behavior

The Docker setup should:

Actual Behavior:


If I use the simple pnpm process, it works well, but I want to set up a Docker container for this so it's always running and I can just open a browser and use it.

Screen Recording / Screenshot

No response

Platform

Additional context

No response

chrismahoney commented 1 day ago
> ❯ docker-compose --profile production up
> WARN[0000] The "GROQ_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "HuggingFace_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "OPEN_ROUTER_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "GOOGLE_GENERATIVE_AI_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "OLLAMA_API_BASE_URL" variable is not set. Defaulting to a blank string.
> WARN[0000] The "GROQ_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "HuggingFace_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "OPEN_ROUTER_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "GOOGLE_GENERATIVE_AI_API_KEY" variable is not set. Defaulting to a blank string.
> WARN[0000] The "OLLAMA_API_BASE_URL" variable is not set. Defaulting to a blank string.
> [+] Running 2/2
>  ✔ Network boltnew-any-llm_default      Created                                                                                                   0.0s
>  ✔ Container boltnew-any-llm-bolt-ai-1  Created                                                                                                   0.1s

@coleam00 @emcconnell @wonderwhy-er Do we pull in #222 here for now, and incur a small bit of tech debt to manage the list of env vars being added to bindings.sh? I think regardless of the solution, we should deal with this sooner than later.

Also @av1155 I see you're using Arc, although that's unrelated to this issue you'll want to be on Chrome Canary while using oTToDev for the time being.

av1155 commented 1 day ago

@chrismahoney I see! Thank you for the tip, will do so!

chrismahoney commented 1 day ago

Also @av1155 for now, please try using the development profile for Docker instead of production. Both should be expected to work obviously, but as this is a relatively recent project (and we don't receive downstream changes from bolt) there will be more immediate updates to dev versus prod. This is more my opinion than anything else.

yamini commented 1 day ago

Same issue here. Specifying the .env.local file was the only thing that worked.

docker compose --env-file .env.local --profile development up --build

av1155 commented 1 day ago

After doing what @yamini said, I was able to load the .env.local variables, but, Ollama still does not work. Pnpm is the only way it will work for me so far as mentioned before.

I just wrote "Hey", and that happened. It thought for a sec, and then broke.

Screenshot 2024-11-20 at 10 37 33 PM

For some reason, the system attempts to use the model model: 'claude-3-5-sonnet-latest', as indicated in the following log, despite the Ollama model being selected. This has been the primary issue I’ve been facing. While I can resolve the .env.local issue by either removing it from .dockerignore or copying the variables manually, I still can't get it to work correctly with Ollama.

Log:

❯ docker compose --env-file .env.local --profile development  up --build
[+] Building 0.4s (12/12) FINISHED                                                                                                docker:desktop-linux
 => [bolt-ai-dev internal] load build definition from Dockerfile                                                                                  0.0s
 => => transferring dockerfile: 2.01kB                                                                                                            0.0s
 => [bolt-ai-dev internal] load metadata for docker.io/library/node:20.18.0                                                                       0.3s
 => [bolt-ai-dev internal] load .dockerignore                                                                                                     0.0s
 => => transferring context: 427B                                                                                                                 0.0s
 => [bolt-ai-dev base 1/5] FROM docker.io/library/node:20.18.0@sha256:a7a3b7ec6de4b11bb2d673b31de9d28c6da09c557ee65453672c8e4f754c23fc            0.0s
 => => resolve docker.io/library/node:20.18.0@sha256:a7a3b7ec6de4b11bb2d673b31de9d28c6da09c557ee65453672c8e4f754c23fc                             0.0s
 => [bolt-ai-dev internal] load build context                                                                                                     0.0s
 => => transferring context: 7.43kB                                                                                                               0.0s
 => CACHED [bolt-ai-dev base 2/5] WORKDIR /app                                                                                                    0.0s
 => CACHED [bolt-ai-dev base 3/5] COPY package.json pnpm-lock.yaml ./                                                                             0.0s
 => CACHED [bolt-ai-dev base 4/5] RUN corepack enable pnpm && pnpm install                                                                        0.0s
 => CACHED [bolt-ai-dev base 5/5] COPY . .                                                                                                        0.0s
 => CACHED [bolt-ai-dev bolt-ai-development 1/1] RUN mkdir -p ${WORKDIR}/run                                                                      0.0s
 => [bolt-ai-dev] exporting to image                                                                                                              0.0s
 => => exporting layers                                                                                                                           0.0s
 => => exporting manifest sha256:7778322d15c9bdfa08c13ecfd508668873ce1a3dfe1963af9d69cca977f7a4f9                                                 0.0s
 => => exporting config sha256:c91758ad11729743b58f8ebe2ab47758b056a0edaed450b13263950789f81c54                                                   0.0s
 => => exporting attestation manifest sha256:8c803e09ac34cf1b13c993b444deba19422a27a184749688ce4c6d9012b4f346                                     0.0s
 => => exporting manifest list sha256:ce4a938bfc2d15f0dd50c613f7af651d0a2df4c687350b586d281436041bee7f                                            0.0s
 => => naming to docker.io/library/bolt-ai:development                                                                                            0.0s
 => => unpacking to docker.io/library/bolt-ai:development                                                                                         0.0s
 => [bolt-ai-dev] resolving provenance for metadata file                                                                                          0.0s
[+] Running 1/0
 ✔ Container boltnew-any-llm-bolt-ai-dev-1  Recreated                                                                                             0.0s
Attaching to bolt-ai-dev-1
bolt-ai-dev-1  |
bolt-ai-dev-1  | > bolt@ dev /app
bolt-ai-dev-1  | > remix vite:dev "--host" "0.0.0.0"
bolt-ai-dev-1  |
bolt-ai-dev-1  |   ➜  Local:   http://localhost:5173/
bolt-ai-dev-1  |   ➜  Network: http://172.19.0.2:5173/
bolt-ai-dev-1  | RetryError [AI_RetryError]: Failed after 3 attempts. Last error: Cannot connect to API: connect ECONNREFUSED 127.0.0.1:11434
bolt-ai-dev-1  |     at _retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:98:13)
bolt-ai-dev-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
bolt-ai-dev-1  |     at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
bolt-ai-dev-1  |     at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
bolt-ai-dev-1  |     at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |     at async chatAction (/app/app/routes/api.chat.ts:64:20)
bolt-ai-dev-1  |     at async Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16)
bolt-ai-dev-1  |     at async /app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4612:21
bolt-ai-dev-1  |     at async callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4677:16)
bolt-ai-dev-1  |     at async Promise.all (index 1)
bolt-ai-dev-1  |     at async callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4552:17)
bolt-ai-dev-1  |     at async callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4041:19)
bolt-ai-dev-1  |     at async submit (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3900:21)
bolt-ai-dev-1  |     at async queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3858:22)
bolt-ai-dev-1  |     at async Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3827:18)
bolt-ai-dev-1  |     at async handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:413:20)
bolt-ai-dev-1  |     at async requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
bolt-ai-dev-1  |     at async /app/node_modules/.pnpm/@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25 {
bolt-ai-dev-1  |   cause: undefined,
bolt-ai-dev-1  |   reason: 'maxRetriesExceeded',
bolt-ai-dev-1  |   errors: [
bolt-ai-dev-1  |     APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED 127.0.0.1:11434
bolt-ai-dev-1  |         at postToApi (/app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.js:446:15)
bolt-ai-dev-1  |         at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
bolt-ai-dev-1  |         at async OllamaChatLanguageModel.doStream (/app/node_modules/.pnpm/ollama-ai-provider@0.15.2_zod@3.23.8/node_modules/ollama-ai-provider/dist/index.js:485:50)
bolt-ai-dev-1  |         at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
bolt-ai-dev-1  |         at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |         at async _retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
bolt-ai-dev-1  |         at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
bolt-ai-dev-1  |         at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
bolt-ai-dev-1  |         at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |         at async chatAction (/app/app/routes/api.chat.ts:64:20)
bolt-ai-dev-1  |         at async Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16)
bolt-ai-dev-1  |         at async /app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4612:21
bolt-ai-dev-1  |         at async callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4677:16)
bolt-ai-dev-1  |         at async Promise.all (index 1)
bolt-ai-dev-1  |         at async callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4552:17)
bolt-ai-dev-1  |         at async callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4041:19)
bolt-ai-dev-1  |         at async submit (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3900:21)
bolt-ai-dev-1  |         at async queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3858:22)
bolt-ai-dev-1  |         at async Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3827:18)
bolt-ai-dev-1  |         at async handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:413:20)
bolt-ai-dev-1  |         at async requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
bolt-ai-dev-1  |         at async /app/node_modules/.pnpm/@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25 {
bolt-ai-dev-1  |       cause: [Error],
bolt-ai-dev-1  |       url: 'http://127.0.0.1:11434/api/chat',
bolt-ai-dev-1  |       requestBodyValues: [Object],
bolt-ai-dev-1  |       statusCode: undefined,
bolt-ai-dev-1  |       responseHeaders: undefined,
bolt-ai-dev-1  |       responseBody: undefined,
bolt-ai-dev-1  |       isRetryable: true,
bolt-ai-dev-1  |       data: undefined,
bolt-ai-dev-1  |       [Symbol(vercel.ai.error)]: true,
bolt-ai-dev-1  |       [Symbol(vercel.ai.error.AI_APICallError)]: true
bolt-ai-dev-1  |     },
bolt-ai-dev-1  |     APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED 127.0.0.1:11434
bolt-ai-dev-1  |         at postToApi (/app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.js:446:15)
bolt-ai-dev-1  |         at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
bolt-ai-dev-1  |         at async OllamaChatLanguageModel.doStream (/app/node_modules/.pnpm/ollama-ai-provider@0.15.2_zod@3.23.8/node_modules/ollama-ai-provider/dist/index.js:485:50)
bolt-ai-dev-1  |         at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
bolt-ai-dev-1  |         at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |         at async _retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
bolt-ai-dev-1  |         at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
bolt-ai-dev-1  |         at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
bolt-ai-dev-1  |         at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |         at async chatAction (/app/app/routes/api.chat.ts:64:20)
bolt-ai-dev-1  |         at async Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16)
bolt-ai-dev-1  |         at async /app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4612:21
bolt-ai-dev-1  |         at async callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4677:16)
bolt-ai-dev-1  |         at async Promise.all (index 1)
bolt-ai-dev-1  |         at async callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4552:17)
bolt-ai-dev-1  |         at async callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4041:19)
bolt-ai-dev-1  |         at async submit (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3900:21)
bolt-ai-dev-1  |         at async queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3858:22)
bolt-ai-dev-1  |         at async Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3827:18)
bolt-ai-dev-1  |         at async handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:413:20)
bolt-ai-dev-1  |         at async requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
bolt-ai-dev-1  |         at async /app/node_modules/.pnpm/@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25 {
bolt-ai-dev-1  |       cause: [Error],
bolt-ai-dev-1  |       url: 'http://127.0.0.1:11434/api/chat',
bolt-ai-dev-1  |       requestBodyValues: [Object],
bolt-ai-dev-1  |       statusCode: undefined,
bolt-ai-dev-1  |       responseHeaders: undefined,
bolt-ai-dev-1  |       responseBody: undefined,
bolt-ai-dev-1  |       isRetryable: true,
bolt-ai-dev-1  |       data: undefined,
bolt-ai-dev-1  |       [Symbol(vercel.ai.error)]: true,
bolt-ai-dev-1  |       [Symbol(vercel.ai.error.AI_APICallError)]: true
bolt-ai-dev-1  |     },
bolt-ai-dev-1  |     APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED 127.0.0.1:11434
bolt-ai-dev-1  |         at postToApi (/app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.js:446:15)
bolt-ai-dev-1  |         at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
bolt-ai-dev-1  |         at async OllamaChatLanguageModel.doStream (/app/node_modules/.pnpm/ollama-ai-provider@0.15.2_zod@3.23.8/node_modules/ollama-ai-provider/dist/index.js:485:50)
bolt-ai-dev-1  |         at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
bolt-ai-dev-1  |         at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |         at async _retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
bolt-ai-dev-1  |         at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
bolt-ai-dev-1  |         at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
bolt-ai-dev-1  |         at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |         at async chatAction (/app/app/routes/api.chat.ts:64:20)
bolt-ai-dev-1  |         at async Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16)
bolt-ai-dev-1  |         at async /app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4612:21
bolt-ai-dev-1  |         at async callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4677:16)
bolt-ai-dev-1  |         at async Promise.all (index 1)
bolt-ai-dev-1  |         at async callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4552:17)
bolt-ai-dev-1  |         at async callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4041:19)
bolt-ai-dev-1  |         at async submit (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3900:21)
bolt-ai-dev-1  |         at async queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3858:22)
bolt-ai-dev-1  |         at async Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3827:18)
bolt-ai-dev-1  |         at async handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:413:20)
bolt-ai-dev-1  |         at async requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
bolt-ai-dev-1  |         at async /app/node_modules/.pnpm/@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25 {
bolt-ai-dev-1  |       cause: [Error],
bolt-ai-dev-1  |       url: 'http://127.0.0.1:11434/api/chat',
bolt-ai-dev-1  |       requestBodyValues: [Object],
bolt-ai-dev-1  |       statusCode: undefined,
bolt-ai-dev-1  |       responseHeaders: undefined,
bolt-ai-dev-1  |       responseBody: undefined,
bolt-ai-dev-1  |       isRetryable: true,
bolt-ai-dev-1  |       data: undefined,
bolt-ai-dev-1  |       [Symbol(vercel.ai.error)]: true,
bolt-ai-dev-1  |       [Symbol(vercel.ai.error.AI_APICallError)]: true
bolt-ai-dev-1  |     }
bolt-ai-dev-1  |   ],
bolt-ai-dev-1  |   lastError: APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED 127.0.0.1:11434
bolt-ai-dev-1  |       at postToApi (/app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.js:446:15)
bolt-ai-dev-1  |       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
bolt-ai-dev-1  |       at async OllamaChatLanguageModel.doStream (/app/node_modules/.pnpm/ollama-ai-provider@0.15.2_zod@3.23.8/node_modules/ollama-ai-provider/dist/index.js:485:50)
bolt-ai-dev-1  |       at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
bolt-ai-dev-1  |       at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |       at async _retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
bolt-ai-dev-1  |       at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
bolt-ai-dev-1  |       at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
bolt-ai-dev-1  |       at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
bolt-ai-dev-1  |       at async chatAction (/app/app/routes/api.chat.ts:64:20)
bolt-ai-dev-1  |       at async Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16)
bolt-ai-dev-1  |       at async /app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4612:21
bolt-ai-dev-1  |       at async callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4677:16)
bolt-ai-dev-1  |       at async Promise.all (index 1)
bolt-ai-dev-1  |       at async callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4552:17)
bolt-ai-dev-1  |       at async callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4041:19)
bolt-ai-dev-1  |       at async submit (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3900:21)
bolt-ai-dev-1  |       at async queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3858:22)
bolt-ai-dev-1  |       at async Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3827:18)
bolt-ai-dev-1  |       at async handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:413:20)
bolt-ai-dev-1  |       at async requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
bolt-ai-dev-1  |       at async /app/node_modules/.pnpm/@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25 {
bolt-ai-dev-1  |     cause: Error: connect ECONNREFUSED 127.0.0.1:11434
bolt-ai-dev-1  |         at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1607:16)
bolt-ai-dev-1  |         at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
bolt-ai-dev-1  |       errno: -111,
bolt-ai-dev-1  |       code: 'ECONNREFUSED',
bolt-ai-dev-1  |       syscall: 'connect',
bolt-ai-dev-1  |       address: '127.0.0.1',
bolt-ai-dev-1  |       port: 11434
bolt-ai-dev-1  |     },
bolt-ai-dev-1  |     url: 'http://127.0.0.1:11434/api/chat',
bolt-ai-dev-1  |     requestBodyValues: {
bolt-ai-dev-1  |       format: undefined,
bolt-ai-dev-1  |       model: 'claude-3-5-sonnet-latest',
bolt-ai-dev-1  |       options: [Object],
bolt-ai-dev-1  |       messages: [Array],
bolt-ai-dev-1  |       tools: undefined
bolt-ai-dev-1  |     },
bolt-ai-dev-1  |     statusCode: undefined,
bolt-ai-dev-1  |     responseHeaders: undefined,
bolt-ai-dev-1  |     responseBody: undefined,
bolt-ai-dev-1  |     isRetryable: true,
bolt-ai-dev-1  |     data: undefined,
bolt-ai-dev-1  |     [Symbol(vercel.ai.error)]: true,
bolt-ai-dev-1  |     [Symbol(vercel.ai.error.AI_APICallError)]: true
bolt-ai-dev-1  |   },
bolt-ai-dev-1  |   [Symbol(vercel.ai.error)]: true,
bolt-ai-dev-1  |   [Symbol(vercel.ai.error.AI_RetryError)]: true
bolt-ai-dev-1  | }
dimanicus commented 1 day ago

I have the same issue with local Llama models (any of them). It may be the problem with parsing Llama model name, 'cause it seems in case of parsing error default model name is used, and it is Claude's 3.5 Sonnet.

It doesn't really explain its inability to connect to port 11434 though (I have just the same issue). May be it's a whole another matter. Because when I tried to change default constants to 'Ollama' and 'qwen2.5-coder:7b-32k', the connect error was still there, only model name in error message changed to qwen one. Here, look:

15:25:10 [vite] page reload app/utils/constants.ts (x2)
RetryError [AI_RetryError]: Failed after 3 attempts. Last error: Cannot connect to API: connect ECONNREFUSED ::1:11434
    at _retryWithExponentialBackoff (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:98:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async startStep (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
    at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
    at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
    at async chatAction (D:/ottodev/bolt.new-any-llm/app/routes/api.chat.ts:64:20)
    at async Object.callRouteAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\data.js:37:16)
    at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4612:21
    at async callLoaderOrAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4677:16)
    at async Promise.all (index 1)
    at async callDataStrategyImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4552:17)
    at async callDataStrategy (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4041:19)
    at async submit (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3900:21)
    at async queryImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3858:22)
    at async Object.queryRoute (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3827:18)
    at async handleResourceRequest (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:413:20)
    at async requestHandler (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:156:18)
    at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge\node_modules\@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
  cause: undefined,
  reason: 'maxRetriesExceeded',
  errors: [
    APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED ::1:11434
        at postToApi (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@ai-sdk+provider-utils@1.0.20_zod@3.23.8\node_modules\@ai-sdk\provider-utils\dist\index.js:446:15)
        at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
        at async OllamaChatLanguageModel.doStream (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\ollama-ai-provider@0.15.2_zod@3.23.8\node_modules\ollama-ai-provider\dist\index.js:485:50)
        at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
        at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
        at async _retryWithExponentialBackoff (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
        at async startStep (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
        at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
        at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
        at async chatAction (D:/ottodev/bolt.new-any-llm/app/routes/api.chat.ts:64:20)
        at async Object.callRouteAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\data.js:37:16)
        at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4612:21
        at async callLoaderOrAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4677:16)
        at async Promise.all (index 1)
        at async callDataStrategyImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4552:17)
        at async callDataStrategy (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4041:19)
        at async submit (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3900:21)
        at async queryImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3858:22)
        at async Object.queryRoute (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3827:18)
        at async handleResourceRequest (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:413:20)
        at async requestHandler (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:156:18)
        at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge\node_modules\@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
      cause: [Error],
      url: 'http://localhost:11434/api/chat',
      requestBodyValues: [Object],
      statusCode: undefined,
      responseHeaders: undefined,
      responseBody: undefined,
      isRetryable: true,
      data: undefined,
      [Symbol(vercel.ai.error)]: true,
      [Symbol(vercel.ai.error.AI_APICallError)]: true
    },
    APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED ::1:11434
        at postToApi (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@ai-sdk+provider-utils@1.0.20_zod@3.23.8\node_modules\@ai-sdk\provider-utils\dist\index.js:446:15)
        at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
        at async OllamaChatLanguageModel.doStream (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\ollama-ai-provider@0.15.2_zod@3.23.8\node_modules\ollama-ai-provider\dist\index.js:485:50)
        at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
        at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
        at async _retryWithExponentialBackoff (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
        at async startStep (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
        at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
        at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
        at async chatAction (D:/ottodev/bolt.new-any-llm/app/routes/api.chat.ts:64:20)
        at async Object.callRouteAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\data.js:37:16)
        at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4612:21
        at async callLoaderOrAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4677:16)
        at async Promise.all (index 1)
        at async callDataStrategyImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4552:17)
        at async callDataStrategy (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4041:19)
        at async submit (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3900:21)
        at async queryImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3858:22)
        at async Object.queryRoute (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3827:18)
        at async handleResourceRequest (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:413:20)
        at async requestHandler (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:156:18)
        at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge\node_modules\@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
      cause: [Error],
      url: 'http://localhost:11434/api/chat',
      requestBodyValues: [Object],
      statusCode: undefined,
      responseHeaders: undefined,
      responseBody: undefined,
      isRetryable: true,
      data: undefined,
      [Symbol(vercel.ai.error)]: true,
      [Symbol(vercel.ai.error.AI_APICallError)]: true
    },
    APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED ::1:11434
        at postToApi (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@ai-sdk+provider-utils@1.0.20_zod@3.23.8\node_modules\@ai-sdk\provider-utils\dist\index.js:446:15)
        at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
        at async OllamaChatLanguageModel.doStream (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\ollama-ai-provider@0.15.2_zod@3.23.8\node_modules\ollama-ai-provider\dist\index.js:485:50)
        at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
        at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
        at async _retryWithExponentialBackoff (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
        at async startStep (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
        at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
        at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
        at async chatAction (D:/ottodev/bolt.new-any-llm/app/routes/api.chat.ts:64:20)
        at async Object.callRouteAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\data.js:37:16)
        at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4612:21
        at async callLoaderOrAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4677:16)
        at async Promise.all (index 1)
        at async callDataStrategyImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4552:17)
        at async callDataStrategy (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4041:19)
        at async submit (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3900:21)
        at async queryImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3858:22)
        at async Object.queryRoute (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3827:18)
        at async handleResourceRequest (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:413:20)
        at async requestHandler (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:156:18)
        at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge\node_modules\@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
      cause: [Error],
      url: 'http://localhost:11434/api/chat',
      requestBodyValues: [Object],
      statusCode: undefined,
      responseHeaders: undefined,
      responseBody: undefined,
      isRetryable: true,
      data: undefined,
      [Symbol(vercel.ai.error)]: true,
      [Symbol(vercel.ai.error.AI_APICallError)]: true
    }
  ],
  lastError: APICallError [AI_APICallError]: Cannot connect to API: connect ECONNREFUSED ::1:11434
      at postToApi (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@ai-sdk+provider-utils@1.0.20_zod@3.23.8\node_modules\@ai-sdk\provider-utils\dist\index.js:446:15)
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async OllamaChatLanguageModel.doStream (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\ollama-ai-provider@0.15.2_zod@3.23.8\node_modules\ollama-ai-provider\dist\index.js:485:50)
      at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
      at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
      at async _retryWithExponentialBackoff (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
      at async startStep (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
      at async fn (file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
      at async file:///D:/ottodev/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
      at async chatAction (D:/ottodev/bolt.new-any-llm/app/routes/api.chat.ts:64:20)
      at async Object.callRouteAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\data.js:37:16)
      at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4612:21
      at async callLoaderOrAction (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4677:16)
      at async Promise.all (index 1)
      at async callDataStrategyImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4552:17)
      at async callDataStrategy (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:4041:19)
      at async submit (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3900:21)
      at async queryImpl (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3858:22)
      at async Object.queryRoute (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+router@1.17.0\node_modules\@remix-run\router\dist\router.cjs.js:3827:18)
      at async handleResourceRequest (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:413:20)
      at async requestHandler (D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules\@remix-run\server-runtime\dist\server.js:156:18)
      at async D:\ottodev\bolt.new-any-llm\node_modules\.pnpm\@remix-run+dev@2.10.0_@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge\node_modules\@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
    cause: Error: connect ECONNREFUSED ::1:11434
        at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1615:16)
        at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
      errno: -4078,
      code: 'ECONNREFUSED',
      syscall: 'connect',
      address: '::1',
      port: 11434
    },
    url: 'http://localhost:11434/api/chat',
    requestBodyValues: {
      format: undefined,
      model: 'qwen2.5-coder:7b-32k',
      options: [Object],
      messages: [Array],
      tools: undefined
    },
    statusCode: undefined,
    responseHeaders: undefined,
    responseBody: undefined,
    isRetryable: true,
    data: undefined,
    [Symbol(vercel.ai.error)]: true,
    [Symbol(vercel.ai.error.AI_APICallError)]: true
  },
  [Symbol(vercel.ai.error)]: true,
  [Symbol(vercel.ai.error.AI_RetryError)]: true
}
jeugregg commented 14 hours ago

same issue than @av1155 And the docker compose --env-file .env.local --profile development up --build from @yamini is working for me in developement of course. I use this link : http://localhost:11434/

ColbyAttack commented 13 hours ago

Same issues I posted about yesterday on the Ottodev Group - hopefully, we can get it figured out. Manually specifying the docker env didn't resolve mine. I still cant seem any of the Ollama models in the dropdown.

coleam00 commented 7 hours ago

@ColbyAttack, @jeugregg, @dimanicus, @av1155 - this issue where it still tries to use Claude when the provider is changed to Ollama is generally fixed by restarting the container. Obviously not ideal but we are still looking into this problem specifically!