OpenGVLab / InternVideo

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding
Apache License 2.0
1.4k stars 85 forks source link

load_state_dict: _IncompatibleKeys(missing_keys=[], unexpected_keys=['temp', 'itm_head.weight', 'itm_head.bias']) #174

Open lexilii opened 2 months ago

lexilii commented 2 months ago

运行 InternVideo2_stage2_1B 的 demo.ipynb 时,intern_model, tokenizer = setup_internvideo2(config) 会报如下警告: load_state_dict: _IncompatibleKeys(missing_keys=[], unexpected_keys=['temp', 'itm_head.weight', 'itm_head.bias'])

最终能够得到如下运行结果: text: A man in a gray sweater plays fetch with his dog in the snowy yard, throwing a toy and watching it run. ~ prob: 0.7927 text: A playful dog and its owner wrestle in the snowy yard, chasing each other with joyous abandon. ~ prob: 0.1769 text: A pet dog excitedly runs through the snowy yard, chasing a toy thrown by its owner. ~ prob: 0.0291 text: A man in a gray hat and coat walks through the snowy yard, carefully navigating around the trees. ~ prob: 0.0006 text: A person dressed in a blue jacket shovels the snow-covered pavement outside their house. ~ prob: 0.0003 text: A person stands on the snowy floor, pushing a sled loaded with blankets, preparing for a fun-filled ride. ~ prob: 0.0002 text: A playful dog slides down a snowy hill, wagging its tail with delight. ~ prob: 0.0001 text: A person in a blue jacket walks their pet on a leash, enjoying a peaceful winter walk among the trees. ~ prob: 0.0001 text: A man in a gray coat walks through the snowy landscape, pulling a sleigh loaded with toys. ~ prob: 0.0000 text: A person bundled up in a blanket walks through the snowy landscape, enjoying the serene winter scenery. ~ prob: 0.0000

请问是否模型加载有什么问题?

leexinhao commented 1 month ago

这个警告是没有问题的,这几个权重只在训练时使用