search

open-webui / open-webui

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
https://openwebui.com
MIT License
45.05k stars 5.5k forks source link

LiteLLM Failed to start after upgrade to v0.1.123 #2037

Closed nijisakai closed 5 months ago

nijisakai commented 5 months ago

Bug Report

Description

Bug Summary: after upgrade to v0.1.123 using docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui, i could not use litellm.

Steps to Reproduce: docker run --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 -e http_proxy=http://172.24.235.74:7890 -e https_proxy=http://172.24.235.74:7890 -e no_proxy=127.0.0.1,localhost --name open-webui --restart always ghcr.io/open-webui/open-webui:main

No WEBUI_SECRET_KEY provided
Generating WEBUI_SECRET_KEY
Loading WEBUI_SECRET_KEY from .webui_secret_key
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_name" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_info" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
INFO:     Started server process [1]
INFO:     Waiting for application startup.

  ___                    __        __   _     _   _ ___ 
 / _ \ _ __   ___ _ __   \ \      / /__| |__ | | | |_ _|
| | | | '_ \ / _ \ '_ \   \ \ /\ / / _ \ '_ \| | | || | 
| |_| | |_) |  __/ | | |   \ V  V /  __/ |_) | |_| || | 
 \___/| .__/ \___|_| |_|    \_/\_/ \___|_.__/ \___/|___|
      |_|                                               

v0.1.123 - building the best open-source AI user interface.      
https://github.com/open-webui/open-webui

INFO:apps.litellm.main:start_litellm_background
INFO:apps.litellm.main:run_background_process
INFO:apps.litellm.main:Executing command: ['litellm', '--port', '14365', '--host', '127.0.0.1', '--telemetry', 'False', '--config', '/app/backend/data/litellm/config.yaml']
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
INFO:apps.litellm.main:Subprocess started successfully.
INFO:     172.24.235.74:5127 - "GET /api/v1/chats/d631bebf-42be-4a18-8cc8-38afa4194006/tags HTTP/1.1" 401 Unauthorized
INFO:apps.litellm.main:Subprocess STDERR: /usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_name" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_info" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
INFO:     Started server process [170]
INFO:     Waiting for application startup.
ERROR:    Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 734, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 610, in __aenter__
    await self._router.startup()
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 711, in startup
    await handler()
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3182, in startup_event
    await initialize(**worker_config)
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3026, in initialize
    ) = await proxy_config.load_config(router=llm_router, config_file_path=config)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 2423, in load_config
    router = litellm.Router(**router_params, semaphore=semaphore)  # type:ignore
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 218, in __init__
    self.set_model_list(model_list)
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2391, in set_model_list
    deployment = self._add_deployment(deployment=deployment)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2455, in _add_deployment
    self.set_client(model=deployment.to_json(exclude_none=True))
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2251, in set_client
    http_client=httpx.AsyncClient(
                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1458, in __init__
    {URLPattern(key): transport for key, transport in mounts.items()}
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1458, in <dictcomp>
    {URLPattern(key): transport for key, transport in mounts.items()}
     ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_utils.py", line 364, in __init__
    raise ValueError(
ValueError: Proxy keys should use proper URL forms rather than plain scheme strings. Instead of "localhost", use "localhost://"

ERROR:    Application startup failed. Exiting.
INFO:apps.litellm.main:
INFO:apps.litellm.main:#------------------------------------------------------------#
INFO:apps.litellm.main:#                                                            #
INFO:apps.litellm.main:#              'I don't like how this works...'               #
INFO:apps.litellm.main:#        https://github.com/BerriAI/litellm/issues/new        #
INFO:apps.litellm.main:#                                                            #
INFO:apps.litellm.main:#------------------------------------------------------------#
INFO:apps.litellm.main:
INFO:apps.litellm.main:Thank you for using LiteLLM! - Krrish & Ishaan
INFO:apps.litellm.main:
INFO:apps.litellm.main:
INFO:apps.litellm.main:
INFO:apps.litellm.main:Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
INFO:apps.litellm.main:
INFO:apps.litellm.main:
INFO:apps.litellm.main:LiteLLM: Proxy initialized with Config, Set models:
INFO:apps.litellm.main:    gpt-4-vision-preview
INFO:apps.litellm.main:Subprocess exited with return code 3
INFO:apps.ollama.main:get_all_models()
INFO:     172.24.235.74:5127 - "GET /ollama/urls HTTP/1.1" 401 Unauthorized
INFO:     172.24.235.74:5127 - "GET /ollama/api/version HTTP/1.1" 200 OK
INFO:     172.24.235.74:5127 - "GET /litellm/api/model/info HTTP/1.1" 401 Unauthorized
INFO:     172.16.188.2:12038 - "GET /_app/immutable/assets/Toaster.ebb080d6.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/assets/0.d0791dc7.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12045 - "GET /_app/immutable/nodes/0.e7a02a39.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /_app/immutable/chunks/globals.7f7f1b26.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/index.d6b34b00.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/assets/dayjs.beb6c5d8.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/navigation.9a07e5b5.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /_app/immutable/assets/2.aa9f4d80.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/assets/Messages.e8d876d7.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12045 - "GET /_app/immutable/assets/index.f15c51aa.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/spread.8a54911c.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/each.b7dbf2c4.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /_app/immutable/chunks/index.6a6efc06.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/Toaster.svelte_svelte_type_style_lang.93775e0f.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/nodes/1.97dcaf87.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/index.1873deba.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/stores.f59ab384.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12045 - "GET /_app/immutable/chunks/index.c5fd21ca.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /_app/immutable/nodes/2.2f43cf5d.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/FileSaver.min.898eb36f.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/index.c96f4261.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/chunks/_commonjsHelpers.de833af9.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12045 - "GET /_app/immutable/chunks/index.f1edc29e.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/index.4e9fd85b.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/index.63ce3869.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/index.9e754bfc.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/chunks/index.545bc149.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/index.1d4c93a7.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/create.d7af4d12.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/chunks/index.1df8947f.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/dayjs.min.056d298b.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12045 - "GET /_app/immutable/chunks/Modal.485dfe88.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /assets/fonts/Arimo-Variable.ttf HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /c/d631bebf-42be-4a18-8cc8-38afa4194006 HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/AdvancedParams.9c68df98.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/Tags.a196c744.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/updater.82b4828f.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12045 - "GET /_app/immutable/chunks/menu-trigger.410969e8.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/chunks/ShareChatModal.d10ea67c.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/index.76879409.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/nodes/3.7c88a8d4.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12038 - "GET /_app/immutable/chunks/index.91ca3596.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12047 - "GET /_app/immutable/chunks/AddFilesPlaceholder.7a46d7d4.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/chunks/Messages.92bc3659.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12048 - "GET /_app/immutable/chunks/Selector.3e10537c.js HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/config HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/v1/auths/ HTTP/1.1" 401 Unauthorized
INFO:     172.16.188.2:12039 - "GET /api/config HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /api/v1/auths/ HTTP/1.1" 401 Unauthorized
INFO:     172.16.188.2:12046 - "GET /_app/immutable/chunks/translation.4c741fc2.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/chunks/translation.ebfc406c.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12039 - "GET /_app/immutable/assets/14.4dfbfe89.css HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /_app/immutable/nodes/14.3647be16.js HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /static/favicon.png HTTP/1.1" 200 OK
INFO:     172.16.188.2:12046 - "GET /assets/fonts/Mona-Sans.woff2 HTTP/1.1" 200 OK
INFO:apps.web.models.auths:authenticate_user: hyinn@live.com
INFO:     172.24.235.74:5146 - "POST /api/v1/auths/signin HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/changelog HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:     172.24.235.74:5146 - "GET /ollama/api/tags HTTP/1.1" 200 OK
INFO:apps.openai.main:get_all_models()
INFO:apps.openai.main:get_all_models()
INFO:     172.24.235.74:5146 - "GET /openai/api/models HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421cb8310>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x780421cb8310>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421cb8310>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/backend/apps/litellm/main.py", line 223, in get_models
    r = requests.request(method="GET", url=f"{url}/models")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421cb8310>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO:     172.24.235.74:5146 - "GET /litellm/api/v1/models HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/v1/modelfiles/ HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/v1/prompts/ HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/v1/documents/ HTTP/1.1" 200 OK
INFO:     172.24.235.74:5146 - "GET /api/v1/chats/tags/all HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:     172.24.235.74:5146 - "GET /ollama/api/tags HTTP/1.1" 200 OK
INFO:     172.24.235.74:5147 - "GET /ollama/api/version HTTP/1.1" 200 OK
INFO:apps.openai.main:get_all_models()
INFO:     172.24.235.74:5146 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO:apps.openai.main:get_all_models()
INFO:     172.24.235.74:5147 - "GET /openai/api/models HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7805a81b90d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7805a81b90d0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7805a81b90d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/backend/apps/litellm/main.py", line 223, in get_models
    r = requests.request(method="GET", url=f"{url}/models")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7805a81b90d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO:     172.24.235.74:5147 - "GET /litellm/api/v1/models HTTP/1.1" 200 OK
INFO:     172.24.235.74:5176 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO:     172.24.235.74:5176 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7805a81a5310>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7805a81a5310>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7805a81a5310>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/backend/apps/litellm/main.py", line 339, in proxy
    r = requests.request(
        ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7805a81a5310>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO:     172.24.235.74:5176 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d2f290>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x780421d2f290>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d2f290>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/backend/apps/litellm/main.py", line 339, in proxy
    r = requests.request(
        ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d2f290>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO:     172.24.235.74:5176 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error
INFO:apps.web.models.auths:authenticate_user: demojustine@outlook.com
INFO:     172.16.188.2:12067 - "POST /api/v1/auths/signin HTTP/1.1" 200 OK
INFO:     172.16.188.2:12067 - "GET /api/changelog HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:     172.16.188.2:12068 - "GET /ollama/api/tags HTTP/1.1" 200 OK
INFO:apps.openai.main:get_all_models()
INFO:apps.openai.main:get_all_models()
INFO:     172.16.188.2:12068 - "GET /openai/api/models HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d3e910>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x780421d3e910>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d3e910>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/backend/apps/litellm/main.py", line 223, in get_models
    r = requests.request(method="GET", url=f"{url}/models")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d3e910>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO:     172.16.188.2:12068 - "GET /litellm/api/v1/models HTTP/1.1" 200 OK
INFO:     172.16.188.2:12068 - "GET /api/v1/modelfiles/ HTTP/1.1" 200 OK
INFO:     172.16.188.2:12068 - "GET /api/v1/prompts/ HTTP/1.1" 200 OK
INFO:     172.16.188.2:12068 - "GET /api/v1/documents/ HTTP/1.1" 200 OK
INFO:     172.16.188.2:12068 - "GET /api/v1/chats/tags/all HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:     172.16.188.2:12068 - "GET /ollama/api/tags HTTP/1.1" 200 OK
INFO:     172.16.188.2:12067 - "GET /ollama/api/version HTTP/1.1" 200 OK
INFO:apps.openai.main:get_all_models()
INFO:     172.16.188.2:12068 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO:apps.openai.main:get_all_models()
INFO:     172.16.188.2:12067 - "GET /openai/api/models HTTP/1.1" 200 OK
ERROR:apps.litellm.main:HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d5a850>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x780421d5a850>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d5a850>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/backend/apps/litellm/main.py", line 223, in get_models
    r = requests.request(method="GET", url=f"{url}/models")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=14365): Max retries exceeded with url: /v1/models (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x780421d5a850>: Failed to establish a new connection: [Errno 111] Connection refused'))
INFO:     172.16.188.2:12067 - "GET /litellm/api/v1/models HTTP/1.1" 200 OK

Environment

Reproduction Details

Confirmation:

Logs and Screenshots

Browser Console Logs: [Include relevant browser console logs, if applicable]

Docker Container Logs: [Include relevant Docker container logs, if applicable]

Screenshots (if applicable): [Attach any relevant screenshots to help illustrate the issue]

Installation Method

[Describe the method you used to install the project, e.g., manual installation, Docker, package manager, etc.]

Additional Information

[Include any additional details that may help in understanding and reproducing the issue. This could include specific configurations, error messages, or anything else relevant to the bug.]

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

nijisakai commented 5 months ago

I could not add litellm through here or use the config.yaml image image

tjbck commented 5 months ago

If I had to make a guess it seems like there was an issue parsing the config.yaml file, could you share them with us?