search

BR-IDL / PaddleViT

:robot: PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+
https://github.com/BR-IDL/PaddleViT
Apache License 2.0
1.22k stars 318 forks source link

Update transformer.py #80

Closed libertatis closed 2 years ago

libertatis commented 2 years ago

add argument attn_head_size to Attention

CLAassistant commented 2 years ago

CLA assistant check
All committers have signed the CLA.

xperzy commented 2 years ago

This PR has errors when running the eval/training. The parameter added causes errors in other part of the code.

Please check the following lines and make sure you have test the code before open an PR: https://github.com/BR-IDL/PaddleViT/blob/0455755bd0e4d75fb0dfd7cecdf67d484f68c6ef/image_classification/ViT/transformer.py#L260

libertatis commented 2 years ago

This PR has errors when running the eval/training. The parameter added causes errors in other part of the code.

Please check the following lines and make sure you have test the code before open an PR:

https://github.com/BR-IDL/PaddleViT/blob/0455755bd0e4d75fb0dfd7cecdf67d484f68c6ef/image_classification/ViT/transformer.py#L260

不好意思，第一次提 PR，我写完后，只单独测试了 Attention，忘了改其他类相关的代码了。我改完代码，测试完，再重新提交~

libertatis commented 2 years ago

这下应该就没有问题了。 我运行的训练/评估的命令如下：

To fine-tuning ViT model performance on Cifar10 with a single GPU

python main_single_gpu.py -cfg=./configs/vit_base_patch16_224.yaml -dataset=cifar10 -batch_size=16 -pretrained=path/to/vit/pretrained/model/vit_base_patch16_224

To evaluate ViT model performance on Cifar10 with a single GPU

python main_single_gpu.py -cfg=./configs/vit_base_patch16_224.yaml -dataset=cifar10 -batch_size=16 -eval -pretrained=path/to/vit/pretrained/model/vit_base_patch16_224

To train ViT model performance on Cifar10 with a single GPU

python main_single_gpu.py -cfg=./configs/vit_base_patch16_224.yaml -dataset=cifar10 -batch_size=32