timeout request from devcontainer

[bwdmr]

Before submitting your bug report

• [ ] I believe this is a bug. I'll try to join the Continue Discord for questions
• [ ] I'm not able to find an open issue that reports the same bug
• [ ] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: macos
- Continue: latest
- IDE: vscode
- Model: codestral
- config.json:

{
  "models": [
    {
      "title": "codestral",
      "model": "codestral",
      "contextLength": 32000,
      "apiBase": "http://host.lima.internal:11434",
      "provider": "ollama"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "tabAutocompleteModel": {
    "title": "codestral",
    "model": "codestral",
    "apiBase": "http://host.lima.internal:11434",
    "provider": "ollama"
  },
  "contextProviders": [
    {
      "name": "code",
      "params": {}
    },
    {
      "name": "docs",
      "params": {}
    },
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    },
    {
      "name": "problems",
      "params": {}
    },
    {
      "name": "folder",
      "params": {}
    },
    {
      "name": "codebase",
      "params": {}
    }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit selected code"
    },
    {
      "name": "comment",
      "description": "Write comments for the selected code"
    },
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    },
    {
      "name": "commit",
      "description": "Generate a git commit message"
    }
  ],
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text"
  }
}

Description

hi i try to use continue.dev and vscode running ollama locally, but fail to connect to the api/chat endpoint. i manage to curl it from the terminal within the devcontainer

curl http://host.docker.internal:11434/api/chat -d '{
  "model": "codestral",
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ]

and

adjusted the config with the apiBase directive as mentioned in the docs:

  "models": [
    {
      "title": "codestral",
      "model": "codestral",
      "contextLength": 32000,
      "apiBase": "http://host.docker.internal:11434",
      "provider": "ollama"
    }
  ],

yet fail to access the service from the continue.dev client as i receive the following error message

request to http://host.docker.internal:11434/api/chat failed, reason: getaddrinfo ENOTFOUND host.docker.internal

To reproduce

cant run ollama in docker due to gpu acel cant access host network from rancher desktop except using default setting host.docker.internal client disregards address cant find server

Log output

No response

[bwdmr]

i checked host again and used the ip address referenced within.

console.ts:137 [Extension Host] Error handling webview message: {
  "msg": {
    "messageId": "fd252e02-f992-4b88-a5bf-ef505c0fcb18",
    "messageType": "llm/streamChat",
    "data": {
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "Hi, why is the sky blue?"
            }
          ]
        }
      ],
      "title": "codestral",
      "completionOptions": {}
    }
  }
}

Error: request to http://192.168.5.2:11434/api/chat failed, reason: connect ETIMEDOUT 192.168.5.2:11434

[fry69]

It looks like you might not really be inside the same Docker container when you curl, otherwise it is hard to explain why /etc/hosts seems to be different, which leads the DNS error and the connection timeout when you configure the IP address.

Anyway, this is not really a Continue problem, IMHO.

(see also Discussion on Discord)

[bwdmr]

repro: devcontainer.json

{
  "build": {
    "dockerfile": "Devcontainer/Dockerfile",
  },
  "extensions": [
      "dbaeumer.vscode-eslint",
      "rvest.vs-code-prettier-eslint",
        "james-yu.latex-workshop",
        "bradlc.vscode-tailwindcss"
    ],
    "workspaceMount": "source=/Users/wibe/Documents/projects,target=/workspace,type=bind",
    "workspaceFolder": "/workspace",
    "containerUser": "user",
    "runArgs": [
        "--network=host",
        "-p=11434:11434"
    ],
    "portsAttributes": {
        "5000": {
            "label": "5001"
        }
    }
}

Dockerfile:

# base
FROM swift:5.10.1-jammy
ENV DEBIAN_FRONTEND=noninteractive
RUN useradd -m -s /bin/bash user -u 501 -g 20
RUN apt-get -y update \
  && apt-get -y upgrade \
  && apt-get install -y \
    git \
    curl \
    wget \
    build-essential \
    software-properties-common \
    tzdata

# swift

# node
RUN curl -fsSL https://deb.nodesource.com/setup_current.x | bash -
RUN apt-get install -y nodejs
RUN npm install -g \
    yarn \
    typescript \
    marked \
    dockerfile-language-server-nodejs \
    typescript-language-server \
    vscode-css-languageserver-bin \
    intelephense \
    eslint \
    prettier \
    gulp-cli \
    serve

# latex
RUN apt-get -y install \
  latexmk \
  texinfo \
  texlive-luatex \
  texlive-latex-extra \
  texlive-bibtex-extra \
  texlive-formats-extra \
  texlive-fonts-extra \
  texlive-lang-all

RUN mkdir /home/user/texmf \
  && curl -k 'https://raw.githubusercontent.com/darwiin/yaac-another-awesome-cv/master/yaac-another-awesome-cv.cls' \
  -o /home/user/texmf/yaac-another-awesome-cv.cls

# tools
RUN apt install -y \
  imagemagick

USER user

# setup nvm
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

# pnpm
ENV SHELL="/bin/bash"
RUN curl -fsSL https://get.pnpm.io/install.sh | sh - 
ENV PATH="/home/user/.local/share/pnpm:${PATH}"

# user related config:
RUN sed -i 's/\(#umask\)/umask/' ~/.profile

[bwdmr]

selfish bump