AttributeError: 'OptLM' object has no attribute 'weight_home'

[pxc3113]

(textgen) C:\Users\35934>python -m flexgen.flex_opt --model facebook/opt-1.3b Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████████████| 685/685 [00:00<?, ?B/s] C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\huggingface_hub\file_download.py:133: UserWarning: huggingface_hub cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\35934.cache\huggingface\hub. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the HF_HUB_DISABLE_SYMLINKS_WARNING environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations. To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development warnings.warn(message) Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████████████| 651/651 [00:00<?, ?B/s] Downloading (…)olve/main/vocab.json: 100%|███████████████████████████████████████████| 899k/899k [00:01<00:00, 808kB/s] Downloading (…)olve/main/merges.txt: 100%|███████████████████████████████████████████| 456k/456k [00:00<00:00, 571kB/s] Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████| 221/221 [00:00<?, ?B/s] Exception in thread Thread-2 (copy_worker_func): Traceback (most recent call last): Exception in thread Thread-3 (copy_worker_func): Traceback (most recent call last): File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 1016, in _bootstrap_inner File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 1016, in _bootstrap_inner model size: 2.443 GB, cache size: 0.398 GB, hidden size (prefill): 0.008 GB init weight... Exception in thread Thread-4 (copy_worker_func): Exception in thread Thread-5 (copy_worker_func): Traceback (most recent call last): Traceback (most recent call last): File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 1016, in _bootstrap_inner File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 1016, in _bootstrap_inner Traceback (most recent call last): self.run() File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 953, in run File "C:\Users\35934\miniconda3\envs\textgen\lib\runpy.py", line 196, in _run_module_as_main self.run() File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 953, in run self.run() File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 953, in run self.run() File "C:\Users\35934\miniconda3\envs\textgen\lib\threading.py", line 953, in run self._target(*self._args, self._kwargs) self._target(*self._args, *self._kwargs) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\pytorch_backend.py", line 880, in copy_worker_func File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\pytorch_backend.py", line 880, in copy_worker_func self._target(self._args, self._kwargs) self._target(*self._args, self._kwargs) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\pytorch_backend.py", line 880, in copy_worker_func File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\pytorch_backend.py", line 880, in copy_worker_func return _run_code(code, main_globals, None, File "C:\Users\35934\miniconda3\envs\textgen\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\flex_opt.py", line 1323, in torch.cuda.set_device(cuda_id) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\torch\cuda__init.py", line 350, in set_device torch.cuda.set_device(cuda_id) torch.cuda.set_device(cuda_id) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\torch\cuda__init__.py", line 350, in set_device File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\torch\cuda__init.py", line 350, in set_device torch.cuda.set_device(cuda_id) run_flexgen(args) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\flex_opt.py", line 1215, in run_flexgen File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\torch\cuda\init__.py", line 350, in set_device torch._C._cuda_setDevice(device) torch._C._cuda_setDevice(device) torch._C._cuda_setDevice(device) AttributeError: module 'torch._C' has no attribute '_cuda_setDevice' AttributeError: module 'torch._C' has no attribute '_cuda_setDevice' AttributeError: module 'torch._C' has no attribute '_cuda_setDevice' torch._C._cuda_setDevice(device) AttributeError: module 'torch._C' has no attribute '_cuda_setDevice' model = OptLM(opt_config, env, args.path, policy) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\flex_opt.py", line 618, in init self.load_weight_stream = torch.cuda.Stream() File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\torch\cuda\streams.py", line 34, in new return super(Stream, cls).new__(cls, priority=priority, kwargs) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\torch\cuda_utils.py", line 46, in err_fn raise RuntimeError( RuntimeError: Tried to instantiate dummy base class Stream Exception ignored in: <function OptLM.del at 0x00000281748FEDD0> Traceback (most recent call last): File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\flex_opt.py", line 1149, in del self.delete_all_weights() File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\flex_opt.py", line 804, in delete_all_weights self.delete_weight(j, 0) File "C:\Users\35934\miniconda3\envs\textgen\lib\site-packages\flexgen\flex_opt.py", line 670, in delete_weight for x in self.weight_home[j].pop(): AttributeError: 'OptLM' object has no attribute 'weight_home'

[SeekPoint]

me too

[XiaomingXu1995]

The key Error is: AttributeError: module 'torch._C' has no attribute '_cuda_setDevice'

you can test the environment by import torch print(torch.cuda.is_available()), if False, run the pip install torch torchvision torchaudio to reinstall the packages.