JasonGUTU
commented
1 month ago Deploying such AI models is difficult. Since the Windows system is very complex and our team has almost no experience with Windows, it is indeed difficult to develop a Windows version.
FurkanGozukara
之前折腾过一次，模型就下了半天时间。总共花了一天时间都没搞定，各种报错。
今天突然想起来又折腾了一下午 所有报错解决了，但这加载模型都用了半个小时，而且没进webui显存就被占用完了，最高直接用到30G显存。 眼看就要看到光明了，结果报错了，显存不足。。。 这个项目是不是不适合win系统？ win用户准备尝试的，建议慎重考虑，浪费时间不说，最后可能一场空。 后期作者有对win做优化的考虑吗？
Loaded model config from [options/SUPIR_v0.yaml] Loaded state_dict from [./ckpts/sd_xl_base_1.0_0.9vae.safetensors] Loaded state_dict from [./ckpts/SUPIR-v0Q.ckpt] Loading vision tower: openai/clip-vit-large-patch14-336 Loading checkpoint shards: 33%|██████████████████▋ | 1/3 [07:14<14:28, 434.29s/it] ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ E:\python\SUPIR\gradio_demo.py:56 in │
│ │
│ 53 │
│ 54 # load LLaVA │
│ 55 if use_llava: │
│ ❱ 56 │ llava_agent = LLavaAgent(LLAVA_MODEL_PATH, device=LLaVA_device, load8bit=args.load │
│ 57 else: │
│ 58 │ llava_agent = None │
│ 59 │
│ │
│ E:\python\SUPIR\llava\llava_agent.py:27 in init │
│ │
│ 24 │ │ │ device_map = 'auto' │
│ 25 │ │ model_path = os.path.expanduser(model_path) │
│ 26 │ │ model_name = get_model_name_from_path(model_path) │
│ ❱ 27 │ │ tokenizer, model, image_processor, context_len = load_pretrained_model( │
│ 28 │ │ │ model_path, None, model_name, device=self.device, device_map=device_map, │
│ 29 │ │ │ load_8bit=load_8bit, load_4bit=load_4bit) │
│ 30 │ │ self.model = model │
│ │
│ E:\python\SUPIR\llava\model\builder.py:103 in load_pretrained_model │
│ │
│ 100 │ │ │ │ model = LlavaMPTForCausalLM.from_pretrained(model_path, low_cpu_mem_usag │
│ 101 │ │ │ else: │
│ 102 │ │ │ │ tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False) │
│ ❱ 103 │ │ │ │ model = LlavaLlamaForCausalLM.from_pretrained(model_path, low_cpu_mem_us │
│ 104 │ else: │
│ 105 │ │ # Load language model │
│ 106 │ │ if model_base is not None: │
│ │
│ D:\anaconda3\envs\supir\lib\site-packages\transformers\modeling_utils.py:2795 in from_pretrained │
│ │
│ 2792 │ │ │ │ mismatched_keys, │
│ 2793 │ │ │ │ offload_index, │
│ 2794 │ │ │ │ error_msgs, │
│ ❱ 2795 │ │ │ ) = cls._load_pretrained_model( │
│ 2796 │ │ │ │ model, │
│ 2797 │ │ │ │ state_dict, │
│ 2798 │ │ │ │ loaded_state_dict_keys, # XXX: rename? │
│ │
│ D:\anaconda3\envs\supir\lib\site-packages\transformers\modeling_utils.py:3123 in │
│ _load_pretrained_model │
│ │
│ 3120 │ │ │ │ ) │
│ 3121 │ │ │ │ │
│ 3122 │ │ │ │ if low_cpu_mem_usage: │
│ ❱ 3123 │ │ │ │ │ new_error_msgs, offload_index, state_dict_index = _load_state_dict_i │
│ 3124 │ │ │ │ │ │ model_to_load, │
│ 3125 │ │ │ │ │ │ state_dict, │
│ 3126 │ │ │ │ │ │ loaded_keys, │
│ │
│ D:\anaconda3\envs\supir\lib\site-packages\transformers\modeling_utils.py:698 in │
│ _load_state_dict_into_meta_model │
│ │
│ 695 │ │ │ state_dict_index = offload_weight(param, param_name, state_dict_folder, stat │
│ 696 │ │ elif not load_in_8bit: │
│ 697 │ │ │ # For backward compatibility with older versions of
accelerate│ │ ❱ 698 │ │ │ set_module_tensor_to_device(model, param_name, param_device, **set_module_kw │ │ 699 │ │ else: │ │ 700 │ │ │ if param.dtype == torch.int8 and param_name.replace("weight", "SCB") in stat │ │ 701 │ │ │ │ fp16_statistics = state_dict[param_name.replace("weight", "SCB")] │ │ │ │ D:\anaconda3\envs\supir\lib\site-packages\accelerate\utils\modeling.py:149 in │ │ set_module_tensor_to_device │ │ │ │ 146 │ │ if value is None: │ │ 147 │ │ │ new_value = old_value.to(device) │ │ 148 │ │ elif isinstance(value, torch.Tensor): │ │ ❱ 149 │ │ │ new_value = value.to(device) │ │ 150 │ │ else: │ │ 151 │ │ │ new_value = torch.tensor(value, device=device) │ │ 152 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 15.99 GiB of which 0 bytes is free. Of the allocated memory 29.82 GiB is allocated by PyTorch, and 448.63 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF