search

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Apache License 2.0
35.53k stars 4.36k forks source link

Gradio web server requires model-worker deps #3397

Open ketsapiwiq opened 2 weeks ago

ketsapiwiq commented 2 weeks ago

Hi, I plan to make a lighter Docker image when only API-based inference is needed. But torch is imported when getting conversation templates. Do you have any opinion on how to separate this?

Thanks.

Here are the logs:

fastchat-gradio-web-server-1  | Traceback (most recent call last):
fastchat-gradio-web-server-1  |   File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
fastchat-gradio-web-server-1  |     return _run_code(code, main_globals, None,
fastchat-gradio-web-server-1  |   File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
fastchat-gradio-web-server-1  |     exec(code, run_globals)
fastchat-gradio-web-server-1  |   File "/app/fastchat/serve/gradio_web_server_multi.py", line 12, in <module>
fastchat-gradio-web-server-1  |     from fastchat.serve.gradio_block_arena_anony import (
fastchat-gradio-web-server-1  |   File "/app/fastchat/serve/gradio_block_arena_anony.py", line 19, in <module>
fastchat-gradio-web-server-1  |     from fastchat.model.model_adapter import get_conversation_template
fastchat-gradio-web-server-1  |   File "/app/fastchat/model/__init__.py", line 1, in <module>
fastchat-gradio-web-server-1  |     from fastchat.model.model_adapter import (
fastchat-gradio-web-server-1  |   File "/app/fastchat/model/model_adapter.py", line 16, in <module>
fastchat-gradio-web-server-1  |     import torch
fastchat-gradio-web-server-1  | ModuleNotFoundError: No module named 'torch'