why bert-base-uncased model set constraint to 0.5, qqp test accuracy only 0.3743

[VpouL]

I download the bert-base-uncased from https://huggingface.co/bert-base-uncased and I execute the main.py like the example

python3 main.py --model_name bert-base-uncased \
                --task_name qqp \
                --ckpt_dir <your HF ckpt directory> \
                --constraint 0.5

But I just get the qqp Test accuracy is only 0.3743

And the log like this 12/13/2022 08:27:08 - INFO - main - Pruned Model MAC: 50.00 % 12/13/2022 08:29:00 - INFO - main - qqp Pruning time (s): 133.1617510318756 12/13/2022 08:34:51 - INFO - main - qqp Test accuracy: 0.3743

[WoosukKwon]

Our work accepts a model already trained on a downstream dataset. The HF BERT model is a pre-trained model which has never been trained for any specific downstream dataset. Please try out with our own checkpoints instead.

[VpouL]

Sorry @WoosukKwon, The Google Drive tip prompts do not have permission to view sharing information for this item.

[xiaowanzi-tju]

@WoosukKwon @VpouL google drive can not visit，can u give a new url?