13/10000 实时翻译 13/10000 Real time Translation: Unable to start API using CUDA 划译 Unable to start API using CUDA

[Mr-Harry]

我将server中的--inference-mode改为了cuda，但是它不能工作，报了一个错误为Traceback (most recent call last): (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/serve/_private/deployment_state.py", line 677, in checkready (ServeController pid=8335) , self._version, self._initialization_latency_s = ray.get( (ServeController pid=8335) ^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper (ServeController pid=8335) return fn(*args, kwargs) (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper (ServeController pid=8335) return func(*args, *kwargs) (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/_private/worker.py", line 2659, in get (ServeController pid=8335) values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout) (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/_private/worker.py", line 871, in get_objects (ServeController pid=8335) raise value.as_instanceof_cause() (ServeController pid=8335) ray.exceptions.RayTaskError(RuntimeError): ray::ServeReplica:default:TexTellerRecServer.initialize_and_get_metadata() (pid=8812, ip=172.17.0.6, actor_id=a6e852be8586acf4078f86fc01000000, repr=<ray.serve._private.replica.ServeReplica:default:TexTellerRecServer object at 0x7fcecb4565a0>) (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/concurrent/futures/_base.py", line 449, in result (ServeController pid=8335) return self.get_result() (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/concurrent/futures/_base.py", line 401, in get_result (ServeController pid=8335) raise self._exception (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/serve/_private/replica.py", line 631, in initialize_and_get_metadata (ServeController pid=8335) raise RuntimeError(traceback.format_exc()) from None (ServeController pid=8335) RuntimeError: Traceback (most recent call last): (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/serve/_private/replica.py", line 609, in initialize_and_get_metadata (ServeController pid=8335) await self._user_callable_wrapper.initialize_callable() (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/serve/_private/replica.py", line 901, in initialize_callable (ServeController pid=8335) await self._call_func_or_gen( (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/ray/serve/_private/replica.py", line 867, in _call_func_or_gen (ServeController pid=8335) result = callable(args, kwargs) (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/autodl-tmp/code/SimpleMath/app.py", line 69, in init (ServeController pid=8335) self.model = self.model.to(inf_mode) if inf_mode != 'cpu' else self.model (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/transformers/modeling_utils.py", line 2905, in to (ServeController pid=8335) return super().to(*args, **kwargs) (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1174, in to (ServeController pid=8335) return self._apply(convert) (ServeController pid=8335) ^^^^^^^^^^^^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/nn/modules/module.py", line 780, in _apply (ServeController pid=8335) module._apply(fn) (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/nn/modules/module.py", line 780, in _apply (ServeController pid=8335) module._apply(fn) (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/nn/modules/module.py", line 780, in _apply (ServeController pid=8335) module._apply(fn) (ServeController pid=8335) [Previous line repeated 1 more time] (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/nn/modules/module.py", line 805, in _apply (ServeController pid=8335) param_applied = fn(param) (ServeController pid=8335) ^^^^^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1160, in convert (ServeController pid=8335) return t.to( (ServeController pid=8335) ^^^^^ (ServeController pid=8335) File "/root/miniconda3/envs/develop/lib/python3.12/site-packages/torch/cuda/init.py", line 314, in _lazy_init (ServeController pid=8335) torch._C._cuda_init() (ServeController pid=8335) RuntimeError: No CUDA GPUs are available.请帮助我