search

qnguyen3 / chat-with-mlx

An all-in-one LLMs Chat UI for Apple Silicon Mac using MLX Framework.
https://twitter.com/stablequan
MIT License
1.48k stars 134 forks source link

openai.InternalServerError: Error code: 503 #117

Closed chameleonscool closed 4 months ago

chameleonscool commented 6 months ago

chat-with-mlx (mlx-chat) 16:45:31

Starting MLX Chat on port 7860 Sharing: False Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`. Fetching 6 files: 100%|██████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 6.34it/s] Traceback (most recent call last): File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/queueing.py", line 527, in process_events response = await route_utils.call_process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/route_utils.py", line 261, in call_process_api output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/blocks.py", line 1786, in process_api result = await self.call_function( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/blocks.py", line 1350, in call_function prediction = await utils.async_iteration(iterator) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/utils.py", line 583, in async_iteration return await iterator.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/utils.py", line 709, in asyncgen_wrapper response = await iterator.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/chat_interface.py", line 545, in _stream_fn first_response = await async_iteration(generator) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/utils.py", line 583, in async_iteration return await iterator.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/utils.py", line 576, in __anext__ return await anyio.to_thread.run_sync( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 851, in run result = context.run(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/gradio/utils.py", line 559, in run_sync_iterator_async return next(iterator) ^^^^^^^^^^^^^^ File "/Users/qian/projects/github/chat-with-mlx/chat_with_mlx/app.py", line 203, in chatbot response = client.chat.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_utils/_utils.py", line 275, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 667, in create return self._post( ^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 1213, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 902, in request return self._request( ^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 978, in _request return self._retry_request( ^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 1026, in _retry_request return self._request( ^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 978, in _request return self._retry_request( ^^^^^^^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 1026, in _retry_request return self._request( ^^^^^^^^^^^^^^ File "/Users/qian/miniconda3/envs/mlx-chat/lib/python3.11/site-packages/openai/_base_client.py", line 993, in _request raise self._make_status_error_from_response(err.response) from None openai.InternalServerError: Error code: 503 Model has loaded, but got this error when ask. Change model got the same error.I was installed with the manual way, using the main branch to install.
rodgomesc commented 6 months ago

same problem, why it tries to connect to openai btw?

qnguyen3 commented 4 months ago

it is now fixed, no more openai client