PINTO0309
commented
4 years ago Thank you for commenting on this for a hobbyist like me who does DeepLearning as a hobby. I am not an engineer or a researcher.
Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)?
No. By the way, I've already successfully converted NCHW to NHWC, but in a very primitive way I did. Since Tensorflow's Conv2D and several other OPs do not support NCHW, this was accomplished by inserting Transpose OPs before and after each OP. While this method can be made to infer correctly, the inserted Transpose OP resulted in unnecessary overhead and a significant loss of original performance. I used a combination of Keras and OpenVINO's model_optimizer to achieve the NHWC to NCHW conversion. (Converting backwards is easy.)
Is there an easy way to convert TF1-pb to TF2-saved_models.pb ?
Yes. I've described how to do the conversion in some of my blog posts below. [English ver.] [Tensorflow Lite] Various Neural Network Model quantization methods for Tensorflow Lite (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization, EdgeTPU). As of May 05, 2020. Alternatively, you can find a repository of perfect tutorials below. https://github.com/leimao/Frozen_Graph_TensorFlow
Is NHWC slowing down execution on the GPU?
I'm not sure. I'm not really interested in using a high performance GPU. I only benchmark with low performance CPUs and edge accelerators. However, I've seen blogs in the past where Japanese engineers have done comparative benchmarking between NCHW and NHWC. However, this article does not refer to the performance of reasoning, but rather shows an increase in learning speed. The improvement in learning speed appears to be from a few percent to a few dozen percent. Japanese article: TensorFlow/Kerasでchannels_firstにするとGPUの訓練が少し速くなる話 - @koshian2
How many FPS do you get on Google Coral TPU-Edge and RaspberryPi4 for yolov4-tiny (int8)?
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 256x256 with 88ms/inference Performance.
Twitter: https://twitter.com/PINTO03091/status/1283429524462428160?s=20
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 416x416 with 243ms/inference Performance.
Twitter: https://twitter.com/PINTO03091/status/1282319908097810437?s=20
Unfortunately, the conversion to models for Coral TPU-Edge was not successful due to a bug in the Tensorflow Lite Converter.
What script did you use to get yolov4_tiny_voc.json?
- First, clone the following repositories. https://github.com/bubbliiiing/yolov4-tiny-keras.git
- All you have to do is modify the
test.pyscript, change it to the following and run it.### tensorflow-gpu==1.15.2 from nets.yolo4_tiny import yolo_body from keras.layers import Input image_input = Input(shape=(416, 416, 3)) model = yolo_body(image_input, 3, 20) model.summary() json_string = model.to_json() open('yolov4_tiny_voc.json', 'w').write(json_string)
AlexeyAB
nihui
itsmasabdi
@PINTO0309 Hi, Nice work with YOLOv4 / tiny!
As I see you use:
NCHW for: OpenVINO (xml / bin), Darknet (cfg / weights)
NHWC for:
TFLite,Keras(yolov4_tiny_voc.json / yolov4_tiny_voc.h5),TF1(pb),TF2(saved_models.json / saved_models.pb)I have several questions:
Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)? I've seen converters that add transpose before and after each layer, but this seems to slow things down a lot. Is it possible to do this transformation without slowing down the inference?
Is there an easy way to convert TF1-pb to TF2-saved_models.pb ?
Is NHWC slowing down execution on the GPU?
How many FPS do you get on
Google Coral TPU-EdgeandRaspberryPi4for yolov4-tiny (int8)?What script did you use to get
yolov4_tiny_voc.json?