503 Service Unavailable

[foolyoghurt]

Here is my User/setting.json in VSCode

{
  "editor.formatOnSave": true,
  "editor.formatOnType": true,
  "[typescriptreact]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "[typescript]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "[javascript]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "search.exclude": {
    "**/.git": true
  },
  "[json]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "editor.formatOnPaste": true,
  "[less]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "cmake.configureOnOpen": true,
  "github.copilot.advanced": {
    "debug.testOverrideProxyUrl": "http://localhost:5001",
    "debug.overrideProxyUrl": "http://localhost:5001"
  }
}

But when I select Mistral Model，it posts to /v1/engines/copilot-codex/completions and get 503 Service Unavailable

Running server...
DEBUG:asyncio:Using selector: KqueueSelector
INFO:     Started server process [25920]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:5001 (Press CTRL+C to quit)
DEBUG:root:Running: python3 -m llama_cpp.server --model /Users/song/models/mistral-7b-instruct-v0.1.Q5_K_M.gguf --n_gpu_layers 1 --n_ctx 4096
INFO:     127.0.0.1:61688 - "POST /set_target HTTP/1.1" 200 OK
Successfully sent selection: Mistral-7b.
DEBUG:root:Current state: {'url': 'https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q5_K_M.gguf', 'type': 'local', 'filename': 'mistral-7b-instruct-v0.1.Q5_K_M.gguf'}
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/Users/song/Codes/venv/localpilot/lib/python3.10/site-packages/certifi/cacert.pem'
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/Users/song/Codes/venv/localpilot/lib/python3.10/site-packages/certifi/cacert.pem'
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/Users/song/Codes/venv/localpilot/lib/python3.10/site-packages/certifi/cacert.pem'
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/Users/song/Codes/venv/localpilot/lib/python3.10/site-packages/certifi/cacert.pem'
DEBUG:httpcore.connection:connect_tcp.started host='127.0.0.1' port=8888 local_address=None timeout=30 socket_options=None
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x103ea65f0>
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'POST']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'POST']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'POST']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 503, b'Service Unavailable', [(b'Connection', b'close'), (b'Proxy-Connection', b'close'), (b'Content-Length', b'8481'), (b'Content-Type', b'text/html;charset=utf-8')])
INFO:httpx:HTTP Request: POST http://localhost:8000/v1/engines/copilot-codex/completions "HTTP/1.1 503 Service Unavailable"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'POST']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
INFO:     127.0.0.1:61693 - "POST /v1/engines/copilot-codex/completions HTTP/1.1" 503 Service Unavailable

[foolyoghurt]

Solved. change the model_folder in config.py, but in app.py, the model_directory = os.path.expanduser(”~/models”) is still hardcoded. That’s why the modified model_folder is not being activated.

[danielgross]

Fixed tyvm