有人成功了吗？

[Vector-Cross]

我试图用我的3060（12G）跑，内存不够，需要什么配置的卡才能成功呀，多卡行不行

我在huggingface上建立了一个空间，MiniGPT-4的docker部署，有没有成功的大佬来指点一下 https://huggingface.co/spaces/zylj/MiniGPT-4

[2132660698]

成功了，比较耗显存。

[Vector-Cross]

怎么17G显存就能跑呀，你这卡也太多了吧

[huangzhongzhong]

卡在准备模型这一步了

[TsuTikgiau]

@huangzhongzhong It looks like the name of your tokenizer is incorrect. It should be 'LlamaTokenizer' instead of 'LLaMATokenizer'. You can update it in the tokenizer config file ' tokenizer_config.json' in your llama weight folder

[Viewerand]

怎么17G显存就能跑呀，你这卡也太多了吧

int4量化了的应该是

[Kizai]

我运行报这个错误

[TsuTikgiau]

Hi @Kizai , from the error info I see the placeholder of vicuna weight '/path/to/vicuna/weights', that means the vicuna weight path is not set. Please check the Readme to know how to set it. Thanks!

[WangRongsheng]

@TsuTikgiau

1. I have set vicuna weights in minigpt4/configs/models/minigpt4.yaml
2. I have set minigpt4 weights in eval_configs/minigpt4_eval.yaml
3. How should the llama weights be configured to load?

[TsuTikgiau]

@WangRongsheng Hello! How do you prepare the vicuna weights? The vicuna weight from huggingface is a delta version and cannot be directly used. We provide a guide PrepareVicuna.md to show you how to prepare the final working vicuna weight. LLAMA weight is used only in this preparation. Thanks!

[Kizai]

I encountered the same problem as the first floor，I'm not worthy

[Vector-Cross]

I encountered the same problem as the first floor，I'm not worthy

yes, we are

[WangRongsheng]

@WangRongsheng Hello! How do you prepare the vicuna weights? The vicuna weight from huggingface is a delta version and cannot be directly used. We provide a guide PrepareVicuna.md to show you how to prepare the final working vicuna weight. LLAMA weight is used only in this preparation. Thanks!

good! I have solved it!

[Kizai]

very vague，numb

[2214962083]

Successfully run on windows 11! It took me 6 hours to complete, and I encountered many problems, which were finally solved.

My pc is rtx4090, and I use vicuna 13B. When uploading pictures and asking questions, it takes up 20G of video memory, and it takes about 5-30 seconds to wait for a reply.

[Vector-Cross]

我在huggingface上建立了一个空间，MiniGPT-4的docker部署，有没有成功的大佬来指点一下 https://huggingface.co/spaces/zylj/MiniGPT-4

[created-Bi]

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

[Vector-Cross]

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

我也没有申请到模型，但是我直接在hugging face上找了别人的，也不知道是不是原始模型参数 https://huggingface.co/decapoda-research

[HunterShenSmzh]

I encountered the same problem as the first floor，I'm not worthy

大佬你是如何解决repo_name问题的呀。我无论是使用相对路径还是绝对路径都会报错。附图：

[Gary-yeh]

I use rtx4090 on windows 11 as well, but I got error message AssertionError: Torch not compiled with CUDA enabled

Loading VIT Loading VIT Done Loading Q-Former Loading Q-Former Done Loading LLAMA

============BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/

Loading checkpoint shards: 0%| | 0/3 [00:08<?, ?it/s] ╭────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\MiniGPT-4\demo.py:60 in
│ ** │ 57 model_config = cfg.model_cfg
│ 58 model_config.device_8bit = args.gpu_id
│ 59 model_cls = registry.get_model_class(model_config.arch)
│ ❱ 60 model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id))
│ 61
│ 62 vis_processor_cfg = cfg.datasets_cfg.cc_sbu_align.vis_processor.train
│ 63 vis_processor = registry.get_processor_class(vis_processor_cfg.name).from_config(vis_pro
│
│ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:243 in from_config
│
│ 240 max_txt_len = cfg.get("max_txt_len", 32)
│ 241 end_sym = cfg.get("end_sym", '\n')
│ 242
│ ❱ 243 model = cls(
│ 244 vit_model=vit_model,
│ 245 q_former_model=q_former_model,
│ 246 img_size=img_size,
│
│ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:90 in init
│
│ 87 self.llama_tokenizer.pad_token = self.llama_tokenizer.eos_token
│ 88
│ 89 if self.low_resource:
│ ❱ 90 self.llama_model = LlamaForCausalLM.from_pretrained(
│ 91 llama_model,
│ 92 torch_dtype=torch.float16,
│ 93 load_in_8bit=True,
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:2795 in │ │ from_pretrained
│
│ 2792 mismatched_keys,
│ 2793 offload_index, │ │ 2794 error_msgs, │ │ ❱ 2795 ) = cls._load_pretrained_model(
│ 2796 model,
│ 2797 state_dict,
│ 2798 loaded_state_dict_keys, # XXX: rename?
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:3124 in
│ _load_pretrained_model
│
│ 3121 )
│ 3122
│ 3123 if low_cpu_mem_usage:
│ ❱ 3124 new_error_msgs, offload_index, state_dict_index = _load_state_dict_i │ 3125 model_to_load,
│ 3126 state_dict,
│ 3127 loaded_keys,
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:706 in
│ _load_state_dict_into_meta_model
│
│ 703 fp16_statistics = None
│ 704
│ 705 if "SCB" not in param_name:
│ ❱ 706 set_module_8bit_tensor_to_device(
│ 707 model, param_name, param_device, value=param, fp16_statistics=fp16_s │ 708 )
│ 709
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\utils\bitsandbytes.py:87 in │ set_module_8bit_tensor_to_device
│
│ 84 if value is None:
│ 85 new_value = old_value.to(device)
│ 86 elif isinstance(value, torch.Tensor):
│ ❱ 87 new_value = value.to(device)
│ 88 else:
│ 89 new_value = torch.tensor(value, device=device)
│ 90
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\torch\cuda__init__.py:239 in _lazy_init │
│ 236 "Cannot re-initialize CUDA in forked subprocess. To use CUDA with "
│ 237 "multiprocessing, you must use the 'spawn' start method")
│ 238 if not hasattr(torch._C, '_cuda_getDeviceCount'):
│ ❱ 239 raise AssertionError("Torch not compiled with CUDA enabled")
│ 240 if _cudart is None:
│ 241 raise AssertionError(
│ 242 "libcudart functions unavailable. It looks like you have a broken build?
╰─────────────────────────────────────────── AssertionError: Torch not compiled with CUDA enabled**

Can someone know how to fix it? Thanks

[WangRongsheng]

81 maybe you can see it!☺️

[Vector-Cross]

I use rtx4090 on windows 11 as well, but I got error message AssertionError: Torch not compiled with CUDA enabled

Loading VIT Loading VIT Done Loading Q-Former Loading Q-Former Done Loading LLAMA

============BUG REPORT===================================

Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/ Loading checkpoint shards: 0%| | 0/3 [00:08<?, ?it/s] ╭────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\MiniGPT-4\demo.py:60 in │ ** │ 57 model_config = cfg.model_cfg │ 58 model_config.device_8bit = args.gpu_id │ 59 model_cls = registry.get_model_class(model_config.arch) │ ❱ 60 model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id)) │ 61 │ 62 vis_processor_cfg = cfg.datasets_cfg.cc_sbu_align.vis_processor.train │ 63 vis_processor = registry.get_processor_class(vis_processor_cfg.name).from_config(vis_pro │ │ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:243 in from_config │ │ 240 max_txt_len = cfg.get("max_txt_len", 32) │ 241 end_sym = cfg.get("end_sym", '\n') │ 242 │ ❱ 243 model = cls( │ 244 vit_model=vit_model, │ 245 q_former_model=q_former_model, │ 246 img_size=img_size, │ │ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:90 in init │ │ 87 self.llama_tokenizer.pad_token = self.llama_tokenizer.eos_token │ 88 │ 89 if self.low_resource: │ ❱ 90 self.llama_model = LlamaForCausalLM.from_pretrained( │ 91 llama_model, │ 92 torch_dtype=torch.float16, │ 93 load_in_8bit=True, │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:2795 in │ │ from_pretrained │ │ 2792 mismatched_keys, │ 2793 offload_index, │ │ 2794 error_msgs, │ │ ❱ 2795 ) = cls._load_pretrained_model( │ 2796 model, │ 2797 state_dict, │ 2798 loaded_state_dict_keys, # XXX: rename? │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:3124 in │ _load_pretrained_model │ │ 3121 ) │ 3122 │ 3123 if low_cpu_mem_usage: │ ❱ 3124 new_error_msgs, offload_index, state_dict_index = _load_state_dict_i │ 3125 model_to_load, │ 3126 state_dict, │ 3127 loaded_keys, │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:706 in │ _load_state_dict_into_meta_model │ │ 703 fp16_statistics = None │ 704 │ 705 if "SCB" not in param_name: │ ❱ 706 set_module_8bit_tensor_to_device( │ 707 model, param_name, param_device, value=param, fp16_statistics=fp16_s │ 708 ) │ 709 │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\utils\bitsandbytes.py:87 in │ set_module_8bit_tensor_to_device │ │ 84 if value is None: │ 85 new_value = old_value.to(device) │ 86 elif isinstance(value, torch.Tensor): │ ❱ 87 new_value = value.to(device) │ 88 else: │ 89 new_value = torch.tensor(value, device=device) │ 90 │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\torch\cudainit.py:239 in _lazy_init │ │ 236 "Cannot re-initialize CUDA in forked subprocess. To use CUDA with " │ 237 "multiprocessing, you must use the 'spawn' start method") │ 238 if not hasattr(torch._C, '_cuda_getDeviceCount'): │ ❱ 239 raise AssertionError("Torch not compiled with CUDA enabled") │ 240 if _cudart is None: │ 241 raise AssertionError( │ 242 "libcudart functions unavailable. It looks like you have a broken build? ╰─────────────────────────────────────────── AssertionError: Torch not compiled with CUDA enabled**

Can someone know how to fix it? Thanks

It seems like pytorch version is cpu version. Maybe change torch version and try again can work.

[created-Bi]

乱码问题，这个要改一下Chrome编码方式？

[ljeff97]

乱码问题，这个要改一下Chrome编码方式？

我也都是乱码，想问下应该怎么解决呀

[WeileiZeng]

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

llamA weights

magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA

vicuna weight magnet

https://huggingface.co/lmsys/vicuna-13b-delta-v0 add to the header

magnet:?xt=urn:btih:a7fac57094561a63d53eed943f904abf24c6969d&dn=Vicuna-13B-HF-fp16-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr.o\ rg%3a1337%2fannounce&tr=udp%2ftracker%2 -udp.gbitt.info%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2f\ tracker.tiny-vps.com%3a6969%2fannounce&tr=udp %3a%2f%2ftracker2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounc\ e&tr=udp%3a%2f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a %2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.org%3a\ 6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a %2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=https%3\ a%2f%2fopentracker.i2p.rocks%3a443%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6\ 969%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2f9.rarbg.com% 3a2810%2fannounce

magnet:?xt=urn:btih:1e0c3dbeefe82483f81bd4e7ea959e4953c8081f&dn=Vicuna-13B-ggml-4bit-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr\ .org%3a1337%2fannounce&tr=udp%2fa%2 .rarbg.com%3a2810%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2ftracker\ 2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p .rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=u\ dp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler .org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker-udp.gbitt.in\ fo%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=https%3a%2f %2fopentracker.i2p.rocks%3a443%2fannounce&tr=udp%3a%2\ f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fann\ ounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net% 3a6969%2fannounce

[WangRongsheng]

乱码问题，这个要改一下Chrome编码方式？

可能你的权重加载错误，你可以参考 https://github.com/Vision-CAIR/MiniGPT-4/issues/81

[mayyalove]

使用的Windows 上面的docker跑的吗？我的会报错

[cnxupupup]

乱码问题，这个要改一下Chrome编码方式？

同样乱码，我配置的是7B的模型，请问如何解决？

[chwshuang]

window10下的经验： git clone https://github.com/Vision-CAIR/MiniGPT-4.git cd MiniGPT-4 conda env create -f environment.yml conda activate minigpt4 pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117 pip uninstall bitsandbytes pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl 其中有2个模型配置修改，和模型权重下载，可以参考：https://github.com/rbbrdckybk/MiniGPT-4 run python demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0