FaceNet_128.tflite

[Sep-AI]

Hi Kao, thank you very much for your repo! I learned a lot about quantisation. My only question is that how you get to convert FaceNet to quant int8? Did you use Quantization-Keras-and-Tensorflow for that? If yes what model you use? Thank you in advanced! Sep

[Kao1126]

Hello, Actually, I didn’t use Quantization-Keras-and-Tensorflow directly. It is the simple tool to convert classified model. I use the code (https://github.com/LeslieZhoa/tensorflow-facenet) and insert the code for quantity-training. After you finished the training, you can convert to the .pb and .thflite.

quantity-training -> .pb -> .tflite -> compile on edge-tpu

You can find the code and progress in the Quantization-Keras-and-Tensorflow. The most important things that you have to check is the operations that you use are supported (https://coral.ai/docs/edgetpu/models-intro/). You can't compile model if you use the operations that aren’t supported !!! About model, you can use any models that you want without SOFTMAX layer.

I suggested that you can find out the API and progress in the Quantization-Keras-and-Tensorflow first. And you will know how to do it.

If you have any questions, please feel free to ask. Thank you !!

Sep-AI notifications@github.com 於 2020年1月21日 週二 上午2:50寫道：

Hi Kao, thank you very much for your repo! I learned a lot about quantisation. My only question is that how you get to convert FaceNet to quant int8? Did you use Quantization-Keras-and-Tensorflow for that? If yes what model you use? Thank you in advanced!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Kao1126/EdgeTPU-FaceNet/issues/1?email_source=notifications&email_token=AMYUC7RFB3MRPNLQZT5LKDLQ6XW65A5CNFSM4KJIQD42YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IHN3ZJQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMYUC7RPTA3KTZAOUYIFZDDQ6XW65ANCNFSM4KJIQD4Q .

[Sep-AI]

Thank you very much for your prompt response! As you said I went and investigate the quantise aware training based on your suggestions and started to work with the repo you introduced i.e. https://github.com/LeslieZhoa/tensorflow-facenet I also find another repo here and try to work around where to add tf.contrib.quantize.create_training_graph() and tf.contrib.quantize.create_eval_graph() but I still receiving some errors. As for tf.contrib.quantize.create_training_graph(), I added it after

prelogits, _ = network.inference(x, keep_probability_placeholder, phase_train=phase_train_placeholder, bottleneck_layer_size=config.embedding_size, weight_decay=config.weight_decay) in train.py

Here is what I've tried for evaluation,

def main():

graph = tf.Graph()
with graph.as_default():
    image_size = (config.image_size, config.image_size)
    with tf.variable_scope('inputs'):
        x = tf.placeholder(dtype=tf.float32, shape=[None, config.image_size, config.image_size,  3])

    batch_size_placeholder = tf.placeholder(tf.int32, name='batch_size')
    phase_train_placeholder = tf.placeholder(tf.bool, name='phase_train')
    image_paths_placeholder = tf.placeholder(tf.string, shape=(None, 1), name='image_paths')
    labels_placeholder = tf.placeholder(tf.int32, shape=(None, 1), name='label')
    keep_probability_placeholder = tf.placeholder(tf.float32, name='keep_probability')

    prelogits, _ = network.inference(x,
                                     keep_probability_placeholder,
                                     phase_train=phase_train_placeholder,
                                     bottleneck_layer_size=config.embedding_size,
                                     weight_decay=config.weight_decay)
    tf.contrib.quantize.create_eval_graph()
    tf.global_variables_initializer()
    saver = tf.train.Saver(sharded=True)
    # import pdb;pdb.set_trace()
    graph.finalize()

    with tf.Session(graph=graph) as session:
        checkpoint = tf.train.latest_checkpoint('/homes/smeshkinfamfard/PycharmProjects/tensorflow-facenet/model/')
        # import pdb;pdb.set_trace()
        saver.restore(session, checkpoint)
        # import pdb;pdb.set_trace()
        builder = tf.saved_model.Builder('exports')

        signature_def = tf.saved_model.predict_signature_def(
            inputs={'x': x},
            outputs=prelogits
        )

        builder.add_meta_graph_and_variables(
            sess=session,
            tags=[
                tf.saved_model.tag_constants.SERVING
            ],
            signature_def_map={
                tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature_def
            },
            saver=saver
        )

        builder.save()

But unfortunately I've encountered an error which is Key InceptionResnetV1/Block8/Conv2d_1x1/act_quant/max not found in checkpoint [[node save/RestoreV2 (defined at quantise.py:22) ]]. This is surprising because I've checked the .meta graph and it does contain InceptionResnetV1/Block8/Conv2d_1x1/act_quant/max but the checkpoint itself doesn't contain this layer which is very odd!

Could you please direct me and show me how you add these two quantise layers. It would be great if you could show me your script for quantisation. Thank you so much in advance!

[Sep-AI]

Hi, Sorry to spam you! here is the repo I've written https://github.com/Sep-AI/FaceNet And here is the latest error I get:

2020-03-31 17:27:45.572601: F tensorflow/lite/toco/graph_transformations/resolve_tensorflow_switch.cc:98] Check failed: other_op->type == OperatorType::kMerge Found BatchNormalization as non-selected output from Switch, but only Merge supported.

It would be so nice of you if you could help me! Many thanks

[Sep-AI]

Hi There, Sorry to disturb you again! This project is really important for me and my future career really depends on this. It would be a great help if you could help me with this. The mail problem as I understands and you can see from my last message is the batch norm layer during the evaluation. Can you please do me a favour and send me your quantization script? I am happy to return your favour as a gift of some sort :)

[Kao1126]

Hello, I am sorry. Because I am really busy recently and I forgot to reply you. I would check the bug and respond you within a few days.

Are you the Chinese or Taiwanese? I saw that there is Chinese in your github.

I can speak Chinese. Maybe we can communicate with it.

On Wed, Apr 15, 2020 at 7:38 AM Sep-AI notifications@github.com wrote:

Reopened #1 https://github.com/Kao1126/EdgeTPU-FaceNet/issues/1.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/Kao1126/EdgeTPU-FaceNet/issues/1#event-3233281418, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMYUC7RXQ623XCJHOK6PD73RMTXZBANCNFSM4KJIQD4Q .

[Kao1126]

MAYBE the problem is because you have batch normalization layer in the model. It would be broken if you quantize directly. You have to add some codes to avoid it. The problem occurs when you use my github directly?

On Sat, Apr 18, 2020 at 12:03 AM Kevin Kao kevinkao1126@gmail.com wrote:

Hello, I am sorry. Because I am really busy recently and I forgot to reply you. I would check the bug and respond you within a few days.

Are you the Chinese or Taiwanese? I saw that there is Chinese in your github.

I can speak Chinese. Maybe we can communicate with it.

On Wed, Apr 15, 2020 at 7:38 AM Sep-AI notifications@github.com wrote:

Reopened #1 https://github.com/Kao1126/EdgeTPU-FaceNet/issues/1.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/Kao1126/EdgeTPU-FaceNet/issues/1#event-3233281418, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMYUC7RXQ623XCJHOK6PD73RMTXZBANCNFSM4KJIQD4Q .

[Sep-AI]

Hi, Thank you so much for your response. I am Iranian so unfortunately I only can speak english. I can use google translate just in case you like to write in Chinese. Yes! you are right. my problem is with Batch Normalisation. I have written my repo here https://github.com/Sep-AI/FaceNet based on your suggestion. I get https://github.com/LeslieZhoa/tensorflow-facenet code and add tf.contrib.quantize.create_training_graph() to line 87 here https://github.com/Sep-AI/FaceNet/blob/master/train/train.py and trained the graph. Then for creating eval graph I used this repo https://github.com/lusinlu/tensorflow_lite_guide/blob/master/quantization_aware_training.py and I write my own code here https://github.com/Sep-AI/FaceNet/blob/master/eval/quant.py But I get the batch normalization error. Please have a look and see if there is any problem with my code. I also have 2 questions. Did you use Keras for training and evaluation of FaceNet_128.tflite? Also is it possible for you to send me the code you wrote for quantity-training -> .pb -> .tflite of FaceNet_128.tflite? I know you are busy so please answer me when you have time. Thanks you very much!