search

changlin31 / BossNAS

(ICCV 2021) BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
137 stars 20 forks source link

imagenet ACC1 is low (49.6%) when evaluate BossNet-T0-80_8.pth #4

Closed fanliaveline closed 2 years ago

fanliaveline commented 3 years ago

Hello! I try to reproduce your model,but when I evaluate the pretrained model(BossNet-T0-80_8.pth),the ACC1 is too low! Did i miss something? Can you help me?

The run command as follows: root@v-dev-11135821-66b7bdd9f5-l9rlv:/data/juicefs_hz_cv_v3/11135821/bak/BossNAS/retraining_hytra# python main.py --model bossnet_T0 --input-size 224 --batch-size 128 --eval --resume /data/juicefs_hz_cv_v3/11135821/bak/model/BossNet-T0-80_8.pth Not using distributed mode Namespace(aa='rand-m9-mstd0.5-inc1', batch_size=128, clip_grad=None, color_jitter=0.4, cooldown_epochs=10, cutmix=1.0, cutmix_minmax=None, data_path='/data/glusterfs_cv_04/public_data/imagenet/CLS-LOC/', data_set='IMNET', decay_epochs=30, decay_rate=0.1, device='cuda', dist_url='env://', distributed=False, drop=0.0, drop_block=None, drop_path=0.1, epochs=300, eval=True, inat_category='name', input_size=224, local_rank=0, lr=0.0005, lr_noise=None, lr_noise_pct=0.67, lr_noise_std=1.0, min_lr=1e-05, mixup=0.8, mixup_mode='batch', mixup_prob=1.0, mixup_switch_prob=0.5, model='bossnet_T0', model_ema=True, model_ema_decay=0.99996, model_ema_force_cpu=False, momentum=0.9, num_workers=10, opt='adamw', opt_betas=None, opt_eps=1e-08, output_dir='output/bossnet_T0-20210804-163815', patience_epochs=10, pin_mem=True, recount=1, remode='pixel', repeated_aug=True, reprob=0.25, resplit=False, resume='/data/juicefs_hz_cv_v3/11135821/bak/model/BossNet-T0-80_8.pth', sched='cosine', seed=0, smoothing=0.1, start_epoch=0, train_interpolation='bicubic', warmup_epochs=5, warmup_lr=1e-06, weight_decay=0.05, world_size=1) Creating model: bossnet_T0 number of params: 38415960 Test: [ 0/261] eta: 0:39:36 loss: 1.9650 (1.9650) acc1: 68.2292 (68.2292) acc5: 90.1042 (90.1042) time: 9.1044 data: 4.8654 max mem: 5605 Test: [ 50/261] eta: 0:01:38 loss: 3.2916 (3.0472) acc1: 41.6667 (47.3039) acc5: 65.1042 (70.5372) time: 0.2928 data: 0.0004 max mem: 5605 Test: [100/261] eta: 0:01:01 loss: 2.9675 (3.1048) acc1: 46.8750 (45.7921) acc5: 69.2708 (69.9722) time: 0.2953 data: 0.0003 max mem: 5605 Test: [150/261] eta: 0:00:39 loss: 2.4230 (2.9457) acc1: 55.2083 (47.8960) acc5: 75.5208 (71.5370) time: 0.2989 data: 0.0003 max mem: 5605 Test: [200/261] eta: 0:00:20 loss: 2.6540 (2.9105) acc1: 48.4375 (48.1913) acc5: 68.7500 (71.3490) time: 0.3023 data: 0.0002 max mem: 5605 Test: [250/261] eta: 0:00:03 loss: 1.6506 (2.8344) acc1: 61.9792 (49.0310) acc5: 83.8542 (72.0431) time: 0.3036 data: 0.0003 max mem: 5605 Test: [260/261] eta: 0:00:00 loss: 1.6135 (2.8001) acc1: 65.6250 (49.5740) acc5: 86.4583 (72.5500) time: 0.3888 data: 0.0001 max mem: 5605 Test: Total time: 0:01:28 (0.3397 s / it)

luke-avionics commented 3 years ago

same issue here

luke-avionics commented 3 years ago

@fanliaveline Hi, have you found any ways to resolve this issue?

changlin31 commented 3 years ago

This is caused by reorganizaiton of the code. I will take a look into it.

changlin31 commented 2 years ago

Hi, @luke-avionics @fanliaveline

This issue is solved in the latest commit bb8b26a09e3ae7889dd118950b022be6b3ce99a0.

The mismatch was here: https://github.com/changlin31/BossNAS/blob/5e8622e580170214ed073d5eb16c5bcdf48f3fa0/retraining_hytra/boss_candidates/bot_op.py#L31 should be https://github.com/changlin31/BossNAS/blob/bb8b26a09e3ae7889dd118950b022be6b3ce99a0/retraining_hytra/boss_candidates/bot_op.py#L30