search

JunkyByte / easy_ViTPose

Easy and fast 2d human and animal multi pose estimation using SOTA ViTPose [Y. Xu et al., 2022] Real-time performances and multiple skeletons supported.
Apache License 2.0
128 stars 19 forks source link

Why the offical model can not match? #36

Closed BOTYW-0 closed 2 weeks ago

BOTYW-0 commented 1 month ago

When i use the wholebody model from the offical vitpose models, it seems not match. More errors like : size mismatch for backbone.blocks.9.attn.qkv.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([2304, 768]). size mismatch for backbone.blocks.9.attn.qkv.bias: copying a param with shape torch.Size([3072]) from checkpoint, the shape in current model is torch.Size([2304]). size mismatch for backbone.blocks.9.attn.proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for backbone.blocks.9.attn.proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for backbone.blocks.9.norm2.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]). size mismatch for backbone.blocks.9.norm2.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]).

JunkyByte commented 1 month ago

Provide more informations. Are you using matching model sizes for checkpoint and model declaration?

BOTYW-0 commented 1 month ago

Provide more informations. Are you using matching model sizes for checkpoint and model declaration?

Yes, i'm sure. I downloaded model Vitpose +-L form the wholebody dataset section, and I renamed this model 'vitpose-l-wholebody.path'

JunkyByte commented 1 month ago

The models are split into the different heads using https://github.com/JunkyByte/easy_ViTPose/blob/main/model_split.py The models provided in hugging face are taken from the original section with this process applied. Try using the model_split script (which was taken from original vitpose repo https://github.com/ViTAE-Transformer/ViTPose/blob/d5216452796c90c6bc29f5c5ec0bdba94366768a/tools/model_split.py#L4)

BOTYW-0 commented 1 month ago

Thanks for your reply! I used the official website's code segmentation model before, but it also reported an error. I downloaded another model and successfully ran it using your segmentation code.Thanks again for your answers.

The models are split into the different heads using https://github.com/JunkyByte/easy_ViTPose/blob/main/model_split.py The models provided in hugging face are taken from the original section with this process applied. Try using the model_split script (which was taken from original vitpose repo https://github.com/ViTAE-Transformer/ViTPose/blob/d5216452796c90c6bc29f5c5ec0bdba94366768a/tools/model_split.py#L4)