search

dqshuai / MetaFormer

A PyTorch implementation of "MetaFormer: A Unified Meta Framework for Fine-Grained Recognition". A reference PyTorch implementation of “CoAtNet: Marrying Convolution and Attention for All Data Sizes”
MIT License
210 stars 36 forks source link

about acc of stanfordcars #20

Open hyao1 opened 2 years ago

hyao1 commented 2 years ago

good work, I achieve perforemance of CUB-200-2011(92.3%), but I only got 93.73% on stanfordcars. Is it any other different configure compared with CUB-200-2011 except for lr?

dqshuai commented 2 years ago

In my experiment, lr had the greatest impact on the experimental results. Try --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 with batchsize 512?

hyao1 commented 2 years ago

In my experiment, lr had the greatest impact on the experimental results. Try --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20?

python3 -m torch.distributed.launch --nproc_per_node 2 --master_port 12335 main.py --cfg ./configs/MetaFG_1_224.yaml --batch-size 32 --tag stcar_v2 --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 --dataset stanfordcars --pretrain ./pretrained_model/metafg_1_inat21_384.pth --accumulation-steps 8 --opts DATA.IMG_SIZE 384.This is my config. I did use the recommended lr. Is it other different with cub-200-2011?

dqshuai commented 2 years ago

In my experiment, lr had the greatest impact on the experimental results. Try --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20?

python3 -m torch.distributed.launch --nproc_per_node 2 --master_port 12335 main.py --cfg ./configs/MetaFG_1_224.yaml --batch-size 32 --tag stcar_v2 --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 --dataset stanfordcars --pretrain ./pretrained_model/metafg_1_inat21_384.pth --accumulation-steps 8 --opts DATA.IMG_SIZE 384.This is my config. I did use the recommended lr. Is it other different with cub-200-2011?

The actual training lr is related to batchsize, specifically actual_lr = lr * total_batchsize/512

hyao1 commented 2 years ago

In my experiment, lr had the greatest impact on the experimental results. Try --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20?

python3 -m torch.distributed.launch --nproc_per_node 2 --master_port 12335 main.py --cfg ./configs/MetaFG_1_224.yaml --batch-size 32 --tag stcar_v2 --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 --dataset stanfordcars --pretrain ./pretrained_model/metafg_1_inat21_384.pth --accumulation-steps 8 --opts DATA.IMG_SIZE 384.This is my config. I did use the recommended lr. Is it other different with cub-200-2011?

The actual training lr is related to batchsize, specifically actual_lr = lr * total_batchsize/512

Recommanded config includes --nproc_per_node 8 --batch-size 32 --accumulation-steps 2. There are only 2*3090 for me, So I modify these into --nproc_per_node 2 --batch-size 32 --accumulation-steps 8 to maintain same total_batchsize. Is it right? I also notad `fused_weight_gradient_mlp_cuda module not found. gradient accumulation fusion with weight gradient computation disabled.in my log file. Dose it mean that--accumulation-steps` dose not work in my code?