search

rgeirhos / texture-vs-shape

Pre-trained models, data, code & materials from the paper "ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness" (ICLR 2019 Oral)
https://openreview.net/forum?id=Bygh9j09KX
Other
785 stars 101 forks source link

For a given image/s, how to predict using model_C = "resnet50_trained_on_SIN_and_IN_then_finetuned_on_IN" ? #14

Closed gaebw closed 4 years ago

gaebw commented 4 years ago
  1. I would like to test/predict model_C on other images ? Is there any code examples?

Step 0 : I was able to load the models.

Step 1: transforms test_images = datasets.ImageFolder( testdir, transforms.Compose([ transforms.Resize(224), transforms.CenterCrop(224), transforms.ToTensor(), normalize, ]))

Step 2: Predict for image in test_images : out = model.forward(image) ? # model.apply(image) ? model(image) ?

  1. model_A, _B, _C are trained with resnet50. Is there any model trained with resnet152 ?

Tnx, Gabi

rgeirhos commented 4 years ago

Hi @gaebw

  1. The provided model is a standard ResNet-50, like the one you'd get from the PyTorch model zoo. I use 
        model.eval()
    input_var = torch.autograd.Variable(input, volatile=True)
    output = model(input_var)

    for inference. If you're not entirely sure whether the predictions are correct you may want to reproduce the reported ImageNet validation accuracies.

  2. The provided models have a ResNet-50 architecture. You may be able to train your own model (e.g. a ResNet-152) using the original dataset (https://github.com/rgeirhos/Stylized-ImageNet).

Does that answer your questions?

gaebw commented 4 years ago

Thanks for your message. Q: The results for cat case, are the following

  1. input image = cat7.png

top 5 classes predicted by the mode: [('n02123159 tiger cat', 0.6769948601722717), ('n02123045 tabby, tabby cat', 0.13011880218982697), ('n02124075 Egyptian cat', 0.039017051458358765), ('n02127052 lynx, catamount', 0.023141874000430107), ('n02129165 lion, king of beasts, Panthera leo', 0.01633445918560028)]

Good cat is detected!

  1. input image = 'cat7-elephant1.png'

[('n02504013 Indian elephant, Elephas maximus', 0.7468844056129456), ('n02504458 African elephant, Loxodonta africana', 0.1721459925174713), ('n01871265 tusker', 0.07487782090902328), ('n02113978 Mexican hairless', 0.003489746479317546), ('n02398521 hippopotamus, hippo, river horse, Hippopotamus amphibius', 0.0017421272350475192)]

Where is the cat?

  1. input image = 'cat7-elephant1_filled.png'

[('n01704323 triceratops', 0.4145205020904541), ('n04599235 wool, woolen, woollen', 0.20820631086826324), ('n02504458 African elephant, Loxodonta africana', 0.045736853033304214), ('n02869837 bonnet, poke bonnet', 0.034652963280677795), ('n04325704 stole', 0.0324515625834465)]

Where is the cat?

  1. input image = 'cat7_silhouette.png'

[('n02104365 schipperke', 0.828610897064209), ('n02124075 Egyptian cat', 0.04147612303495407), ('n02133161 American black bear, black bear, Ursus americanus, Euarctos americanus', 0.02275703474879265), ('n02096585 Boston bull, Boston terrier', 0.016802627593278885), ('n02085620 Chihuahua', 0.012340801768004894)]

Where is the cat?

Is there any model less sensitive to texture? that is, detect all the above images as a cat?

Tnx!

rgeirhos commented 4 years ago

model_C corresponds to a model that is fine-tuned on ImageNet. It is therefore still quite sensitive to texture (model_A has the strongest shape bias). Still, there is no guarantee that this model will reliably detect all of the mentioned images as a cat. Does this answer your question?

rgeirhos commented 4 years ago

Closing this issue due to inactivity. Please feel free to re-open if necessary!