tensorflow mobilenet-v2 support

[cjerry1243]

Hi. Since tensorflow has released mobilnet-v2 weights, I'm wondering that is this toolkit goinig to support mobilenet-v2 from tf to caffe?

[JiahaoYao]

@cjerry1243 The padding in Caffe is always symmetric, defined in official tutorial and mentioned in stackoverflow. According to Caffe #6294, there is no non-symmetric padding layer implemented.

Currently, mobilenet v2 conversion from tensorflow to caffe does not work because of this padding problem. Take the first conv layer as examples. It takes the input of 224x224x3 and outputs 112x112x32, with kernel size of 3 and stride 2. In tensorflow, the padding is same, which actually means padding_left=0, padding_right=1, padding_top=0, padding_bottom=1. However, in caffe, the padding is symmetric. Even though p_h = 1, p_w = 1 ( or padding_left=1, padding_right=1, padding_top=1, padding_bottom=1) can also make the output 112x112x32 shape, the value of the output can be different because of mismatch in convolution arithmetic computation. To illustrate this more clearly, I draw the following charts.

Despite intrinsic problem in caffe, we have already add mobilenet-v2 inside MMDNN. You can convert tensorflow mobilenet-v2 into any other frameworks. You can follow the ways described below for the conversion.

---

First, try the newest version by

pip install -U git+https://github.com/Microsoft/MMdnn.git@master

Next, You can refer Slim Model Extractor to extract mobilenet-v2 model,

mmdownload -f tensorflow -n mobilenet_v2_1.0_224

Meta File Graph Visualization

I recommend you to use this tool to visualize the graph

mmvismeta imagenet_mobilenet_v2_1.0_224.ckpt.meta ./logs/

Tensorflow -> IR

You can use following bash command to convert the checkpoint files to IR architecture

mmtoir -f tensorflow -d mobilenet-v2 -n imagenet_mobilenet_v2_1.0_224.ckpt.meta -w imagenet_mobilenet_v2_1.0_224.ckpt --dstNodeName MMdnn_Output

You will get IR architecture file and IR weights file

IR network structure is saved as [mobilenet-v2.json].
IR network structure is saved as [mobilenet-v2.pb].
IR weights are saved as [mobilenet-v2.npy].

IR -> Tensorflow

Convert models from IR to Tensorflow code snippet

mmtocode -f tensorflow --IRModelPath mobilenet-v2.pb --IRWeightPath mobilenet-v2.npy --dstModelPath tf_mobile-v2.py

You will get the tensorflow code

Target network code snippet is saved as [tf_mobile-v2.py].

IR -> Keras

Convert models from IR to Keras code snippet and you will get the keras code

mmtocode -f keras --IRModelPath mobilenet-v2.pb --IRWeightPath mobilenet-v2.npy --dstModelPath tf_mobile-v2.py

Also, it can be converted to coreml and mxnet.

[cjerry1243]

@JiahaoYao Thank you. I have solved padding problem by using pad=2 followed by 2 slice layers in caffe.

[ujsyehao]

@cjerry1243 Hi, Can you explain the details?

[cjerry1243]

@ujsyehao when pad=1, caffe will pad left and right simultaneously, but TF will pad right first, and then pad left if necessary. As the above example, input size is 224, and kernel=3, stride=2 in conv layer. Although output size is the same which is 112, kernels slide differently because they starts from different pixels in the image.

So, the solution is that use pad=2 followed by 2 slice layers in caffe. kernels would slide:

The second kernel starts from the pixel where tf pad=1 kernel starts. Except for the first kernel, the other kernels slide as tf kernels do. The output size become 113, so we need to slice the left column and the top row.

[ujsyehao]

@cjerry1243 Hi, Thank for your solution, I have tested it, it performs well! @JiahaoYao The solution(from cjerry1243) is a good solution, you can use it to do mobilenet v2 conversion(from tensorflow to caffe).

[cjerry1243]

I have finished mobilenet v2 conversion from tf to caffe. It's now on my github.

[ujsyehao]

@cjerry1243 I see your project, Relu6 layer in caffe is easy to implement, you can finish it!

[ChaoYu123]

Hi I wonder how to do the cropping in caffe.NetSpec to get the sliced layers.