windows系统部署的一些问题

[theoldsong]

之前折腾过一次，模型就下了半天时间。总共花了一天时间都没搞定，各种报错。

今天突然想起来又折腾了一下午 所有报错解决了，但这加载模型都用了半个小时，而且没进webui显存就被占用完了，最高直接用到30G显存。 眼看就要看到光明了，结果报错了，显存不足。。。 这个项目是不是不适合win系统？ win用户准备尝试的，建议慎重考虑，浪费时间不说，最后可能一场空。 后期作者有对win做优化的考虑吗？

Loaded model config from [options/SUPIR_v0.yaml] Loaded state_dict from [./ckpts/sd_xl_base_1.0_0.9vae.safetensors] Loaded state_dict from [./ckpts/SUPIR-v0Q.ckpt] Loading vision tower: openai/clip-vit-large-patch14-336 Loading checkpoint shards: 33%|██████████████████▋ | 1/3 [07:14<14:28, 434.29s/it] ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ E:\python\SUPIR\gradio_demo.py:56 in │ │ │ │ 53 │ │ 54 # load LLaVA │ │ 55 if use_llava: │ │ ❱ 56 │ llava_agent = LLavaAgent(LLAVA_MODEL_PATH, device=LLaVA_device, load8bit=args.load │ │ 57 else: │ │ 58 │ llava_agent = None │ │ 59 │ │ │ │ E:\python\SUPIR\llava\llava_agent.py:27 in init │ │ │ │ 24 │ │ │ device_map = 'auto' │ │ 25 │ │ model_path = os.path.expanduser(model_path) │ │ 26 │ │ model_name = get_model_name_from_path(model_path) │ │ ❱ 27 │ │ tokenizer, model, image_processor, context_len = load_pretrained_model( │ │ 28 │ │ │ model_path, None, model_name, device=self.device, device_map=device_map, │ │ 29 │ │ │ load_8bit=load_8bit, load_4bit=load_4bit) │ │ 30 │ │ self.model = model │ │ │ │ E:\python\SUPIR\llava\model\builder.py:103 in load_pretrained_model │ │ │ │ 100 │ │ │ │ model = LlavaMPTForCausalLM.from_pretrained(model_path, low_cpu_mem_usag │ │ 101 │ │ │ else: │ │ 102 │ │ │ │ tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False) │ │ ❱ 103 │ │ │ │ model = LlavaLlamaForCausalLM.from_pretrained(model_path, low_cpu_mem_us │ │ 104 │ else: │ │ 105 │ │ # Load language model │ │ 106 │ │ if model_base is not None: │ │ │ │ D:\anaconda3\envs\supir\lib\site-packages\transformers\modeling_utils.py:2795 in from_pretrained │ │ │ │ 2792 │ │ │ │ mismatched_keys, │ │ 2793 │ │ │ │ offload_index, │ │ 2794 │ │ │ │ error_msgs, │ │ ❱ 2795 │ │ │ ) = cls._load_pretrained_model( │ │ 2796 │ │ │ │ model, │ │ 2797 │ │ │ │ state_dict, │ │ 2798 │ │ │ │ loaded_state_dict_keys, # XXX: rename? │ │ │ │ D:\anaconda3\envs\supir\lib\site-packages\transformers\modeling_utils.py:3123 in │ │ _load_pretrained_model │ │ │ │ 3120 │ │ │ │ ) │ │ 3121 │ │ │ │ │ │ 3122 │ │ │ │ if low_cpu_mem_usage: │ │ ❱ 3123 │ │ │ │ │ new_error_msgs, offload_index, state_dict_index = _load_state_dict_i │ │ 3124 │ │ │ │ │ │ model_to_load, │ │ 3125 │ │ │ │ │ │ state_dict, │ │ 3126 │ │ │ │ │ │ loaded_keys, │ │ │ │ D:\anaconda3\envs\supir\lib\site-packages\transformers\modeling_utils.py:698 in │ │ _load_state_dict_into_meta_model │ │ │ │ 695 │ │ │ state_dict_index = offload_weight(param, param_name, state_dict_folder, stat │ │ 696 │ │ elif not load_in_8bit: │ │ 697 │ │ │ # For backward compatibility with older versions of accelerate │ │ ❱ 698 │ │ │ set_module_tensor_to_device(model, param_name, param_device, **set_module_kw │ │ 699 │ │ else: │ │ 700 │ │ │ if param.dtype == torch.int8 and param_name.replace("weight", "SCB") in stat │ │ 701 │ │ │ │ fp16_statistics = state_dict[param_name.replace("weight", "SCB")] │ │ │ │ D:\anaconda3\envs\supir\lib\site-packages\accelerate\utils\modeling.py:149 in │ │ set_module_tensor_to_device │ │ │ │ 146 │ │ if value is None: │ │ 147 │ │ │ new_value = old_value.to(device) │ │ 148 │ │ elif isinstance(value, torch.Tensor): │ │ ❱ 149 │ │ │ new_value = value.to(device) │ │ 150 │ │ else: │ │ 151 │ │ │ new_value = torch.tensor(value, device=device) │ │ 152 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 15.99 GiB of which 0 bytes is free. Of the allocated memory 29.82 GiB is allocated by PyTorch, and 448.63 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[JasonGUTU]

Deploying such AI models is difficult. Since the Windows system is very complex and our team has almost no experience with Windows, it is indeed difficult to develop a Windows version.

[FurkanGozukara]

we have fully optimized Windows gradio version

full tutorial here : https://youtu.be/OYxVEvDf284

[Jandown]

之前也是花了大量时间没搞定，不甘心，又花了两天时间，终于成功了，结果卡在step2了，大概率是显存不够了 估计也只有3090/4090能跑的动了，其他卡就别浪费时间了