some weights are newly initialied

[bingwork]

import torch from transformers import AutoTokenizer, AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained('internlm/internlm-xcomposer2-vl-7b', torch_dtype=torch.bfloat16, trust_remote_code=True).cuda() You are using a model of type internlmxcomposer2 to instantiate a model of type internlm. This is not supported for all configurations of models and can yield errors. [2024-02-02 01:35:48,326] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect) Set max length to 4096 Position interpolate from 24x24 to 35x35 Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:09<00:00, 4.96s/it] Some weights of InternLMXComposer2ForCausalLM were not initialized from the model checkpoint at internlm/internlm-xcomposer2-vl-7b and are newly initialized: ['vit.vision_tower.vision_model.post_layernorm.weight', 'vit.vision_tower.vision_model.post_layernorm.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

internlm-xcomposer2-vl-7b is really great! when load the model, I saw some weights are newly initialied, and they are not in https://huggingface.co/internlm/internlm-xcomposer2-vl-7b/blob/main/pytorch_model.bin.index.json, I'm not quite sure, could you help to take a look? thank you very much!

@myownskyW7 @LightDXY @eltociear @yhcao6 @vansin

[myownskyW7]

@bingwork Don't worry. The 'vit.vision_tower.vision_model.post_layernorm.weight' and 'vit.vision_tower.vision_model.post_layernorm.bias' are unnecessary in our model. They will be removed later.

[chaochen1998]

Hello, will these messages influence the final performance of the model?

Set max length to 4096
Position interpolate from 24x24 to 35x35

[LightDXY]

hi, this is a simple notification of ViT resolution increasing, we will clean it in the following update.