model.LLM.llama_qformerv2_peft.Transformer can not load

[WeiXuanLi-1024]

when I run the script "alpacaLlava_llamaQformerv2Peft_QF_13B.sh" this error occurred :
`dist_backend='nccl') [09:51:49.180219] Start initialization. [09:51:49.180394] ## Processing on RANK 0. [09:51:49.180664] <class 'model.LLM.llama_qformerv2_peft.Transformer'> [09:51:49.189299] Model Args: ModelArgs(dim=4096, n_layers=32, n_heads=32, n_kv_heads=None, vocab_size=32000, multiple_of=256, ffn_dim_multiplier=None, norm_eps=1e-05, rope_theta=10000, max_batch_size=32, max_seq_len=512, rope_scaling=None, lora_rank=16, bias_tuning=True) [09:52:32.216106] rope theta: 10000 [09:52:32.217181] build llama model with qformerv2 Traceback (most recent call last): File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection raise err File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection sock.connect(sa) OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/connectionpool.py", line 714, in urlopen httplib_response = self._make_request( File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/connectionpool.py", line 403, in _make_request self._validate_conn(conn) File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn conn.connect() File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/connection.py", line 363, in connect self.sock = conn = self._new_conn() File "/home/liwx/anaconda3/envs/accessory/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fb338344df0>: Failed to establish a new connection: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurr`

I test the network ： PING www.a.shifen.com (153.3.238.102) 56(84) bytes of data. 64 bytes from 153.3.238.102 (153.3.238.102): icmp_seq=1 ttl=53 time=3.47 ms 64 bytes from 153.3.238.102 (153.3.238.102): icmp_seq=2 ttl=53 time=3.73 ms 64 bytes from 153.3.238.102 (153.3.238.102): icmp_seq=3 ttl=53 time=3.46 ms

I need climb over the wall to download this model?

[ChrisLiu6]

Our multi-modal models need pre-trained weights for the visual encoder. The following code downloads the pre-trained blip2 model from HuggingFace https://github.com/Alpha-VLLM/LLaMA2-Accessory/blob/f6a1525f59e6c16a87258734ceeb1b06d1b948b6/accessory/model/LLM/llama_qformerv2_peft.py#L291

Please check if you can access huggingface, or proxy may be needed.

[WeiXuanLi-1024]

Our multi-modal models need pre-trained weights for the visual encoder. The following code downloads the pre-trained blip2 model from HuggingFace

https://github.com/Alpha-VLLM/LLaMA2-Accessory/blob/f6a1525f59e6c16a87258734ceeb1b06d1b948b6/accessory/model/LLM/llama_qformerv2_peft.py#L291

Please check if you can access huggingface, or proxy may be needed.

Thanks for your reply, I have solved this problem by downloading offline mode from huggingface but I have another problem, I want to Fine-Tuning with Quantization refer to link https://llama2-accessory.readthedocs.io/en/latest/finetune/quantization.html ,the script alpacaLlava_llamaQformerv2Peft_QF_13B.sh pretrained_path points to download from here stage_1 checkpoint https://huggingface.co/Alpha-VLLM/LLaMA2-Accessory/tree/main/finetune/mm/caption_llamaQformerv2_13b when i run the exps ,this error occurred :

[kriskrisliu]

The error shows that the model you've loaded is not compatible to the checkpoint. Please have a look in Line 5, pass llama2-13B param.json file to the flag: llama_config="<path/to/llama2-13B/param.json> configs/model/finetune/sg/llamaPeft_normBiasLora.json" In this way you will init a 13B-like model to load the 13B checkpoint.

[WeiXuanLi-1024]

this link is right for llama2-13B param.json file ? https://huggingface.co/Alpha-VLLM/LLaMA2-Accessory/blob/main/config/13B_params.json

I have one more question what is difference between alpacaLlava_llamaQformerv2Peft_13b and alpacaLlava_llamaQformerv2_13b ?

[ChrisLiu6]

this link is right for llama2-13B param.json file ? https://huggingface.co/Alpha-VLLM/LLaMA2-Accessory/blob/main/config/13B_params.json

I have one more question what is difference between alpacaLlava_llamaQformerv2Peft_13b and alpacaLlava_llamaQformerv2_13b ?

1. Yes, this is the config file for llama2-13B
2. Both alpacaLlava_llamaQformerv2Peft_13b and alpacaLlava_llamaQformerv2_13b and fine-tuned from caption_llamaQformerv2_13b, but the former is trained with parameter efficient fine-tuning (PEFT), specifically norm/bias/lora tuning, via the latter is with full-parameter tuning.

I notice that you have changed the following line in your script

llama_config="$2 configs/model/finetune/sg/llamaPeft_normBiasLora.json"

You seem to have erased the $2 part. This is incorrect. llama_config is expected to consist of two params.json files in this exp. The first designates the size of the model (e.g. 7B, 13B), and the second defines the PEFT configurations. See the doc for details.

llama_config in .sh to the model parameter file (usually named as params.json) that differentiates model sizes (7B, 13B, …)

You can also assign multiple .json config files to llama_config simultaneously. The combined configuration from all these files will be used, with keys from later files overwriting those from earlier ones. This is especially handy when you want to make specific model configuration adjustments, like the LoRA dimension, which is consistent across various model sizes, eliminating the need to produce individual files for each size.

[WeiXuanLi-1024]

By modifying this script, the exps has worked normally. Thanks again