指定cuda报错

[Lzzzii10]

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)

[JeffRody]

Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 622, in process_events response = await route_utils.call_process_api( File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 323, in call_process_api output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 2016, in process_api result = await self.call_function( File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1569, in call_function prediction = await anyio.to_thread.run_sync( # type: ignore File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2405, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 914, in run result = context.run(func, args) File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 846, in wrapper response = f(args, kwargs) File "/home/wanglch/projects/Qwen2-Audio/demo/web_demo_audio.py", line 93, in predict generate_ids = model.generate(inputs, max_length=256) File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/generation/utils.py", line 2207, in generate result = self._sample( File "/usr/local/lib/python3.10/site-packages/transformers/generation/utils.py", line 3172, in _sample outputs = self(model_inputs, return_dict=True) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/usr/local/lib/python3.10/site-packages/accelerate/hooks.py", line 170, in new_forward output = module._old_forward(args, kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2_audio/modeling_qwen2_audio.py", line 1214, in forward outputs = self.language_model( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 1188, in forward outputs = self.model( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 889, in forward causal_mask = self._update_causal_mask( File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 1009, in _update_causal_mask causal_mask = self._prepare_4d_causal_attention_mask_with_cache_position( File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 1095, in _prepare_4d_causal_attention_mask_with_cache_position padding_mask = causal_mask[:, :, :, :mask_length] + attention_mask[:, None, None, :] RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

same error

[Zero-Captain]

same error