Add EfficientNet-lite Models?

[Path-A]

EfficientNet-lite are a set of mobile/IoT friendly image classification models. Notably, while EfficientNet-EdgeTPU that is specialized for Coral EdgeTPU, these EfficientNet-lite models run well on all mobile CPU/GPU/EdgeTPU.

Due to the requirements from edge devices, we mainly made the following changes based on the original EfficientNets.

• Remove squeeze-and-excite (SE): SE are not well supported for some mobile accelerators.
• Replace all swish with RELU6: for easier post-quantization.
• Fix the stem and head while scaling models up: for keeping models small and fast.

https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite

[galaktyk]

I did try convert lite_0 model to tfkeras model here

⚠ input normalization is (image/127.5) -1.

[seefun]

I did try convert lite_0 model to tfkeras model here Got inference time around 15ms on RTX2070 and 17ms on web tfjs with Xiaomi8. This model seems very optimized for the mobile.

Great job!

[HugoDel]

I really appreciate someone converts the model from tf but I lose the ability to tune my input_shape without having a model class. Does someone know how to change the input_shape/layer? Otherwise is it plan to have a model class for Efficientnet lite?

[galaktyk]

I really appreciate someone converts the model from tf but I lose the ability to tune my input_shape without having a model class. Does someone know how to change the input_shape/layer? Otherwise is it plan to have a model class for Efficientnet lite?

With Keras layer you can do whatever you want.

# load model.
old_model = tf.keras.models.load_model('L0.h5')

# save model weights.
old_model.save_weights('w_only.h5')

# reshape first layer
old_model._layers[0]._batch_input_shape = (None, 96, 128, 3)

# need to re-create model to apply changed in the architecture.
json_model = old_model.to_json()
del old_model
new_model = tf.keras.models.model_from_json(json_model)

# load weights
new_model.load_weights('w_only.h5')

# save new model.
new_model.save('new_model.h5')

[mylyu]

I really appreciate someone converts the model from tf but I lose the ability to tune my input_shape without having a model class. Does someone know how to change the input_shape/layer? Otherwise is it plan to have a model class for Efficientnet lite?

With Keras layer you can do whatever you want.

# load model.
old_model = tf.keras.models.load_model('L0.h5')

# save model weights.
old_model.save_weights('w_only.h5')

# reshape first layer
old_model._layers[0]._batch_input_shape = (None, 96, 128, 3)

# need to re-create model to apply changed in the architecture.
json_model = old_model.to_json()
del old_model
new_model = tf.keras.models.model_from_json(json_model)

# load weights
new_model.load_weights('w_only.h5')

# save new model.
new_model.save('new_model.h5')

@galaktyk thanks for sharing, but have you validated the converted model? I tried some simple images and the predicted classes were wrong...

[galaktyk]

Did you normalized with (image/127.5) -1. ?

[mylyu]

Yes, I did yet results were all close to zeros on all classes. KERAS version problem? I tried tf.keras in tf 2.2 and 1.15.

On Wed, 27 May 2020 at 20:11, galaktyk notifications@github.com wrote:

Did you normalized with (image/127.5) -1. ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qubvel/efficientnet/issues/104#issuecomment-634617897, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACN3SST3L6KGAZZ7XA3NXCDRTT7OHANCNFSM4LFBP3EQ .

[mylyu]

I am using https://upload.wikimedia.org/wikipedia/commons/f/fe/Giant_Panda_in_Beijing_Zoo_1.JPG and got top 5 Predicted: [[('n04501370', 'turnstile', 0.0046648816), ('n02108089', 'boxer', 0.0037645926), ('n04325704', 'stole', 0.0035586052), ('n07583066', 'guacamole', 0.0034703321), ('n04357314', 'sunscreen', 0.0033295394)]]

On Wed, 27 May 2020 at 20:35, Lyu Mengye lvmengye@gmail.com wrote:

Yes, I did yet results were all close to zeros on all classes. KERAS version problem? I tried tf.keras in tf 2.2 and 1.15.

On Wed, 27 May 2020 at 20:11, galaktyk notifications@github.com wrote:

Did you normalized with (image/127.5) -1. ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qubvel/efficientnet/issues/104#issuecomment-634617897, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACN3SST3L6KGAZZ7XA3NXCDRTT7OHANCNFSM4LFBP3EQ .

[galaktyk]

I am using https://upload.wikimedia.org/wikipedia/commons/f/fe/Giant_Panda_in_Beijing_Zoo_1.JPG and got top 5 Predicted: [[('n04501370', 'turnstile', 0.0046648816), ('n02108089', 'boxer', 0.0037645926), ('n04325704', 'stole', 0.0035586052), ('n07583066', 'guacamole', 0.0034703321), ('n04357314', 'sunscreen', 0.0033295394)]] … On Wed, 27 May 2020 at 20:35, Lyu Mengye @.> wrote: Yes, I did yet results were all close to zeros on all classes. KERAS version problem? I tried tf.keras in tf 2.2 and 1.15. On Wed, 27 May 2020 at 20:11, galaktyk @.> wrote: > Did you normalized with (image/127.5) -1. ? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#104 (comment)>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/ACN3SST3L6KGAZZ7XA3NXCDRTT7OHANCNFSM4LFBP3EQ > . >

Got it. I forgot the weights for the L0 model. 🙇‍♂️🙇‍♂️🙇‍♂️ you can download the new one in the drive.

[l0stpenguin]

@galaktyk I tried loading the efficientnet-lite models using keras but it throws an error.

from keras.models import load_model
load_model('L0.h5')

Using TensorFlow backend.
Traceback (most recent call last):
  File "update_l0.py", line 2, in <module>
    load_model('L0.h5')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/saving.py", line 492, in load_wrapper
    return load_function(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/saving.py", line 584, in load_model
    model = _deserialize_model(h5dict, custom_objects, compile)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/saving.py", line 274, in _deserialize_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/saving.py", line 627, in model_from_config
    return deserialize(config, custom_objects=custom_objects)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/layers/__init__.py", line 168, in deserialize
    printable_module_name='layer')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/utils/generic_utils.py", line 147, in deserialize_keras_object
    list(custom_objects.items())))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/network.py", line 1056, in from_config
    process_layer(layer_data)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/network.py", line 1042, in process_layer
    custom_objects=custom_objects)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/layers/__init__.py", line 168, in deserialize
    printable_module_name='layer')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/utils/generic_utils.py", line 149, in deserialize_keras_object
    return cls.from_config(config['config'])
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/base_layer.py", line 1179, in from_config
    return cls(**config)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'ragged'

According to this stackoverflow thread https://stackoverflow.com/questions/58878421/unexpected-keyword-argument-ragged-in-keras, it seems that your model was saved using tensorflow.keras. I have a working keras only (not tensorflow.keras) project, is it possible for me to use your l0 model?

[mylyu]

thanks a lot, this one works.

On Fri, 29 May 2020 at 13:20, galaktyk notifications@github.com wrote:

I am using https://upload.wikimedia.org/wikipedia/commons/f/fe/Giant_Panda_in_Beijing_Zoo_1.JPG and got top 5 Predicted: [[('n04501370', 'turnstile', 0.0046648816), ('n02108089', 'boxer', 0.0037645926), ('n04325704', 'stole', 0.0035586052), ('n07583066', 'guacamole', 0.0034703321), ('n04357314', 'sunscreen', 0.0033295394)]] … <#m4066650546388792615> On Wed, 27 May 2020 at 20:35, Lyu Mengye @.> wrote: Yes, I did yet results were all close to zeros on all classes. KERAS version problem? I tried tf.keras in tf 2.2 and 1.15. On Wed, 27 May 2020 at 20:11, galaktyk @.> wrote: > Did you normalized with (image/127.5) -1. ? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#104 (comment) https://github.com/qubvel/efficientnet/issues/104#issuecomment-634617897>,

or unsubscribe > https://github.com/notifications/unsubscribe-auth/ACN3SST3L6KGAZZ7XA3NXCDRTT7OHANCNFSM4LFBP3EQ . >

Got it. I forgot the weights for the L0 model. 🙇‍♂️🙇‍♂️🙇‍♂️ you can download the new one in the drive.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qubvel/efficientnet/issues/104#issuecomment-635765656, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACN3SSTRC6EO6FXMPDAAHALRT5A2XANCNFSM4LFBP3EQ .

[galaktyk]

@l0stpenguin It's a tensorflow.keras model... ...But you can convert tfkeras to keras anyway.

1. In TFKeras, Load and save tfkeras in .json model with model.to_json()
2. save weights with model.save_weights()
2. edit json file to make it usable in keras
        - remove ragged field
        - change GlorotUniform to RandomUniform
        - Change axis in batchnorm from [3] to 3
4. in Keras, Load .json file and load model with keras.models.model_from_json()
5. load weights

👌 check the link again, I've just uploaded L0 model with keras2.3

[l0stpenguin]

@galaktyk Thanks, i can now load the model. I tried reshaping the input using the snippet you provided but i get this error: AttributeError: 'InputLayer' object has no attribute '_batch_input_shape'

It occurs here: old_model._layers[0]._batch_input_shape = (None, 480,640, 3) Is it possible to reshape the lite model?

[galaktyk]

@l0stpenguin yes. Tested with tfkeras1.14, tfkeras2.0, tfkeras2.1, tfkeras2.2, Keras2.3(backend tf1.14), Keras2.3(backend tf2.0), Keras2.3(backend tf2.1), Keras2.3(backend tf2.2)

[l0stpenguin]

@galaktyk Do you also have the other pretrained lite models for keras (L1,L2,L3)? In the google drive, i can find only L0 for keras but for the tensorflow-keras there are the variants. Please share them if you already have them.

[metanav]

@galaktyk Thanks for sharing this. Do you have any idea how to use it with qubvel/segmentation_models?

[kells1986]

@galaktyk This is awesome and has saved me a lot of time. Do you have the code you used to create these h5 files? It would be amazing if you could share it.