simicon
commented
1 year ago hi there, I also encountered the same mistake, how did you solve it?
Closed disconnectome closed 1 year ago
simicon
commented
1 year ago hi there, I also encountered the same mistake, how did you solve it?
Trying to run a model using wsl2 in win 11. Enviroment installed correctly. Cuda driver is working nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Tue_Mar__8_18:18:20_PST_2022 Cuda compilation tools, release 11.6, V11.6.124 Build cuda_11.6.r11.6/compiler.31057947_0
import torch print(torch.rand(1, device="cuda")) tensor([0.7875], device='cuda:0')
But when I run bot.py, I got (OpenChatKit) disconnectome@disconnectome:~/OpenChatKit$ python inference/bot.py --model togethercomputer/Pythia-Chat-Base-7B Loading togethercomputer/Pythia-Chat-Base-7B to cuda:0... Traceback (most recent call last): File "/home/disconnectome/OpenChatKit/inference/bot.py", line 285, in
main()
File "/home/disconnectome/OpenChatKit/inference/bot.py", line 281, in main
).cmdloop()
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/cmd.py", line 105, in cmdloop
self.preloop()
File "/home/disconnectome/OpenChatKit/inference/bot.py", line 127, in preloop
self._model = ChatModel(self._model_name_or_path, self._gpu_id, self._max_memory)
File "/home/disconnectome/OpenChatKit/inference/bot.py", line 59, in init
self._model.to(device)
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/site-packages/torch/nn/modules/module.py", line 989, in to
return self._apply(convert)
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641, in _apply
module._apply(fn)
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641, in _apply
module._apply(fn)
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641, in _apply
module._apply(fn)
[Previous line repeated 1 more time]
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/site-packages/torch/nn/modules/module.py", line 664, in _apply
param_applied = fn(param)
File "/home/disconnectome/miniconda3/envs/OpenChatKit/lib/python3.10/site-packages/torch/nn/modules/module.py", line 987, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!
Any help please?