search

hahnyuan / PTQ4ViT

Post-Training Quantization for Vision transformers.
182 stars 26 forks source link

How to load the quantized models with PTQ4ViT into the net? #7

Open uniqzheng opened 2 years ago

uniqzheng commented 2 years ago

Hi! Thanks for your great work! There was a problem when I was trying to load the quantized model 'vit_base_patch16_384.pth' (you've provided) into the net created by timm.create_model. The error is as follows.

RuntimeError: Error(s) in loading state_dict for VisionTransformer: Missing key(s) in state_dict: "cls_token", "pos_embed", "patch_embed.proj.weight", "patch_embed.proj.bias", "blocks.0.norm1.weight", "blocks.0.norm1.bias", "blocks.0.attn.qkv.weight", "blocks.0.attn.qkv.bias", "blocks.0.attn.proj.weight", "blocks.0.attn.proj.bias", "blocks.0.norm2.weight", "blocks.0.norm2.bias", "blocks.0.mlp.fc1.weight", "blocks.0.mlp.fc1.bias", "blocks.0.mlp.fc2.weight", "blocks.0.mlp.fc2.bias".......

Would you please provide the correct method to load your quantized model? Thank you a lot~

SuperVan-Young commented 2 years ago

The quantized models only contains NN parameters in int8, which cannot be loaded and put into use directly. I would suggest you follow the given example files and quantize a model from given FP32 ones, which doesn't take too much time in our experiment setting.

SuperVan-Young commented 2 years ago

I've only taken notice of the issues recently. I'll start working on a more user-friendly interface ASAP, so that you don't have to quantize a model from scratch .

uniqzheng commented 2 years ago

Many thanks!!

Wuyy-fairy commented 1 year ago

Excellent work! But Ialso want to know when will the quantized model can be loaded by myself ? Looking forward your reply, thank you and have a good work!