search

anonymous-sushi-armadillo / fast_is_better_than_free_imagenet

10 stars 4 forks source link

The pre-trained model you released is not robust #2

Open TianQi-777 opened 4 years ago

TianQi-777 commented 4 years ago

I downloaded the model weight you released in Google drive folder and found that it is not robust. 

CUDA_VISIBLE_DEVICES=0,1,2,3 python3 main_fast.py ../../1_Dataset/ImageNet/ --config ./configs/configs_fast_2px_evaluate.yml --output_prefix eval_2px --resume ./checkpoint/imagenet_model_weights_2px.pth.tar --evaluate --restarts 1

output

/home/2_class_level/ImageNet/utils.py:81: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  config = EasyDict(yaml.load(f))
INFO:root:***************************** LOGISTICS *****************************
INFO:root:Experiment Date: 2020-08-09 08:21
INFO:root:Output Name: eval_2px_step2_eps2_repeat1
INFO:root:User: XX
INFO:root:***************************** ARGUMENTS *****************************
INFO:root:TRAIN: {'arch': 'resnet50', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0001, 'print_freq': 10, 'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225], 'start_epoch': 12, 'epochs': 15, 'lr_epochs': [12, 15], 'lr_values': [0.004, 0.0004], 'half': True, 'random_init': True}
INFO:root:ADV: {'clip_eps': 2.0, 'fgsm_step': 2.0, 'n_repeats': 1, 'pgd_attack': [(50, 0.00392156862)]}
INFO:root:DATA: {'workers': 4, 'max_color_value': 255.0, 'img_size': 0, 'batch_size': 128, 'crop_size': 288}
INFO:root:data: ../../1_Dataset/ImageNet/
INFO:root:output_prefix: eval_2px
INFO:root:config: ./configs/configs_fast_2px_evaluate.yml
INFO:root:resume: ./checkpoint/imagenet_model_weights_2px.pth.tar
INFO:root:evaluate: True
INFO:root:pretrained: False
INFO:root:restarts: 1
INFO:root:output_name: eval_2px_step2_eps2_repeat1
INFO:root:**********************************************************************
INFO:root:=> creating model 'resnet50'
INFO:root:=> loading checkpoint './checkpoint/imagenet_model_weights_2px.pth.tar'
INFO:root:configs.TRAIN.start_epoch = 15
INFO:root:best_prec1 = 60.89999771118164
INFO:root:=> loaded checkpoint './checkpoint/imagenet_model_weights_2px.pth.tar' (epoch 15)
INFO:root:*********************** Performing PGD Attacks ***********************
INFO:root:******* PGD eps: 2.0, K: 50, step: 0.00392156862, restarts: 1 *******
PGD Test: [0/391]       Time 28.806 (28.806)    Loss 16.8124 (16.8124)  Prec@1 0.000 (0.000)    Prec@5 0.000 (0.000)
PGD Test: [10/391]      Time 12.807 (14.247)    Loss 20.1095 (18.0751)  Prec@1 0.000 (0.000)    Prec@5 0.000 (0.000)
PGD Test: [20/391]      Time 12.812 (13.567)    Loss 21.4244 (18.4928)  Prec@1 0.000 (0.000)    Prec@5 0.000 (0.000)
LexaTang commented 3 years ago

You should resize your dataset. Try resize.py.