TFlite conversion

[ljun901527]

Dear Ranftl,

Your new small model was converted from Pytorch to TFlite directly. It seems that this method doesn't need converting to onnx and PB first. Can you please elaborate on how to realize it?

Best regards! Liu Jun

[AlexeyAB]

@ljun901527 Hi,

It seems that this method doesn't need converting to onnx and PB first.

Yes.

Can you please elaborate on how to realize it?

This converter is private. We used an approach similar to this one (there is an example of a converter to convert an EfficientNet-Lite Classifier/Encoder from Pytorch to TF): https://github.com/rwightman/gen-efficientnet-pytorch/issues/50#issuecomment-707821534

[ljun901527]

@ljun901527 Hi,

It seems that this method doesn't need converting to onnx and PB first.

Yes.

Can you please elaborate on how to realize it?

This converter is private. We used an approach similar to this one (there is an example of a converter to convert an EfficientNet-Lite Classifier/Encoder from Pytorch to TF): rwightman/gen-efficientnet-pytorch#50 (comment)

Thanks for your reply!

[ljun901527]

@ljun901527 Hi,

It seems that this method doesn't need converting to onnx and PB first.

Yes.

Can you please elaborate on how to realize it?

This converter is private. We used an approach similar to this one (there is an example of a converter to convert an EfficientNet-Lite Classifier/Encoder from Pytorch to TF): rwightman/gen-efficientnet-pytorch#50 (comment)

Dear AlexeyAB,

I have followed your method and converted my model from pytorch to tensorflow yielding the same outputs successfully. However, their outputs are different when I firstly initialize the model from checkpoints using "load_state_dict" and then convert the model to tensorflow. And, if my model has not norm layer, this issue does not comes. Do you know how this comes?

[AlexeyAB]

@ljun901527 Hi,

• We are using load_state_dict for networks with batch-normalization and this does not lead to any issues.

And, if my model has not norm layer, this issue does not comes.

• What is the norm layer, is it batch-normalization? Have you created your own network without batch normalization, or have you fused the Convolutional and Batch-norm layers into one layer by using torch.nn.utils.fuse_conv_bn_eval or torch.quantization.fuse_modules ?

• Do you use tf.get_collection_ref(tf.GraphKeys.GLOBAL_VARIABLES) instead of tf.get_collection_ref(tf.GraphKeys.TRAINABLE_VARIABLES) for loading weights to TF model? https://colab.research.google.com/gist/AlexeyAB/cc05f2690c3707d5e0f66d1e749b5546/weights_torch_to_tf_effnet_lite0.ipynb#scrollTo=69xMuwZJBp4e

[ljun901527]

@ljun901527 Hi,

• We are using load_state_dict for networks with batch-normalization and this does not lead to any issues.

And, if my model has not norm layer, this issue does not comes.

• What is the norm layer, is it batch-normalization? Have you created your own network without batch normalization, or have you fused the Convolutional and Batch-norm layers into one layer by using torch.nn.utils.fuse_conv_bn_eval or torch.quantization.fuse_modules ?
• Do you use tf.get_collection_ref(tf.GraphKeys.GLOBAL_VARIABLES) instead of tf.get_collection_ref(tf.GraphKeys.TRAINABLE_VARIABLES) for loading weights to TF model? https://colab.research.google.com/gist/AlexeyAB/cc05f2690c3707d5e0f66d1e749b5546/weights_torch_to_tf_effnet_lite0.ipynb#scrollTo=69xMuwZJBp4e

Thanks for your answer, the results are almost same after using GLOBAL_VARIABLES instead of TRAINABLE_VARIABLES in loading weights function.

[OwenHua666]

@ljun901527 I am stuck at the conversion step for the midasv2.1 small model even though followed the script in the colab. Were you able to convert the midasV2.1 model? Also, @AlexeyAB the colab conversion for efficientNet works for me, but when I need to run the nn.sequential(modules). The inference failed because of the Unet Stucture with skip connection. model = nn.Sequential(model.pretrained.layer1, model.pretrained.layer2, model.pretrained.layer3, model.pretrained.layer4, model.scratch.layer1_rn, model.scratch.layer2_rn, model.scratch.layer3_rn, model.scratch.layer4_rn, model.scratch.activation, model.scratch.refinenet4, model.scratch.refinenet3, model.scratch.refinenet2, model.scratch.refinenet1, model.scratch.output_conv) Do you guys know the correct way to sequential the model blocks for the midas V2.1 small?

[ljun901527]

@ljun901527 I am stuck at the conversion step for the midasv2.1 small model even though followed the script in the colab. Were you able to convert the midasV2.1 model? Also, @AlexeyAB the colab conversion for efficientNet works for me, but when I need to run the nn.sequential(modules). The inference failed because of the Unet Stucture with skip connection. model = nn.Sequential(model.pretrained.layer1, model.pretrained.layer2, model.pretrained.layer3, model.pretrained.layer4, model.scratch.layer1_rn, model.scratch.layer2_rn, model.scratch.layer3_rn, model.scratch.layer4_rn, model.scratch.activation, model.scratch.refinenet4, model.scratch.refinenet3, model.scratch.refinenet2, model.scratch.refinenet1, model.scratch.output_conv) Do you guys know the correct way to sequential the model blocks for the midas V2.1 small?

Sorry, I have only converted my model through this method, and have not converted the midasV2.1 model.

[OwenHua666]

@AlexeyAB By using the example you provided, I encounter a problem when I convert the graph with the upsampling layers and skip connection. Can you provide some help/guidance here?

[fricc33]

@OwenHua666 Did you ever manage to get the conversion to work? Would you share the results?