search

zkkli / PSAQ-ViT

[ECCV 2022] Patch Similarity Aware Data-Free Quantization for Vision Transformers
Apache License 2.0
120 stars 13 forks source link

How long will the data generation take? #1

Open youdutaidi opened 2 years ago

youdutaidi commented 2 years ago

Thank you for sharing your brilliant work ~Could you share the information about your total training time of every stage, especially about the data generating time

WhereKey commented 2 years ago

Yeah, I have the same question about PSAQ-ViT v2. There is no auxiliary category prior for data generation. I think the generator needs much more iterations to learn the differences of all categories.

zkkli commented 2 years ago

Hi,

The whole process takes less than 4 min on an RTX 3090 GPU and most time is spent in the image generation stage, since the parameter calibration without training produces small overhead.

For instance, the specific timing of each stage on DeiT-B model is as follows:

WhereKey commented 2 years ago

I have to say your work is very nice. Most data-free quantization codes(even published on CVPR) lost quantization and do not fully use low-precision input and low-precision weight for matrix multiplication. But your work is to do it completely.👍🏻 I've learned a lot from your codes!