qubvel / efficientnet

Implementation of EfficientNet model. Keras and TensorFlow Keras.
https://arxiv.org/abs/1905.11946
Apache License 2.0
2.07k stars 472 forks source link

EfficientNet version 1.0.0 contains lower accuracy than 0.0.4 #80

Open super-hsu opened 5 years ago

super-hsu commented 5 years ago

With the included example/inference_example.ipynb, I am seeing lower prediction with version 1.0.0 as opposed to version 0.0.4.

Using the below code for testing both versions (with the exception of import efficientnet vs import efficientnet.keras/tfkeras) produce different prediction results:

!pip install efficientnet==0.0.4 !wget https://github.com/qubvel/efficientnet/raw/master/misc/panda.jpg from keras.applications.imagenet_utils import decode_predictions import numpy as np import matplotlib.pyplot as plt from efficientnet import EfficientNetB0 from efficientnet import center_crop_and_resize, preprocess_input

model = EfficientNetB0(weights='imagenet') image = plt.imread('panda.jpg')

image_size = model.input_shape[1] x = center_crop_and_resize(image, image_size=image_size) x = preprocess_input(x) x = np.expand_dims(x, 0)

y = model.predict(x) decode_predictions(y)

Output from version 0.0.4: [[('n02510455', 'giant_panda', 0.8347928), ('n02134084', 'ice_bear', 0.015602051), ('n02509815', 'lesser_panda', 0.0045535257), ('n02133161', 'American_black_bear', 0.002471913), ('n02132136', 'brown_bear', 0.0020707587)]]

Output from version 1.0.0: [[('n02510455', 'giant_panda', 0.75878686), ('n02134084', 'ice_bear', 0.008354747), ('n02132136', 'brown_bear', 0.0072072297), ('n02509815', 'lesser_panda', 0.0041302275), ('n02120079', 'Arctic_fox', 0.0040210797)]]

At its current state (version 1.0.0), I am also seeing lower top-1 score training on CIFAR10 on EfficientNet B0 as opposed to the same on ResNet50, thinking that the issue may be caused by outdated/bad weights as shown above.