ysh329 commented 4 years ago

Caffe模型先转TensorFlow

pip install mmdnn

mmconvert -sf caffe -in -iw -df tensorflow -om

里面包含了 ```shell |-- saved_model.pb `-- variables |-- variables.data-00000-of-00001 `-- variables.index ```

ysh329 commented 4 years ago

caffe转tensorflow的部分log如下


------------------------------------------------------------
    WARNING: PyCaffe not found!
    Falling back to a pure protocol buffer implementation.
    * Conversions will be drastically slower.
    * This backend is UNTESTED!
------------------------------------------------------------

Warning: parameters not reshaped for node: [BatchNorm] face_landmark_337
Warning: parameters not reshaped for node: [BatchNorm] face_landmark_453
Warning: parameters not reshaped for node: [BatchNorm] face_landmark_480
Warning: parameters not reshaped for node: [BatchNorm] face_landmark_505

Type                 Name                                          Param               Output
----------------------------------------------------------------------------------------------
Input                data                                             --     (1, 3, 128, 128)
Convolution          face_landmark_322                   (3, 3, 3, 8)       (1, 8, 64, 64)
BatchNorm            face_landmark_337                           (8,)       (1, 8, 64, 64)
ReLU                 face_landmark_339                             --       (1, 8, 64, 64)
Convolution          face_landmark_438                   (3, 3, 1, 8)       (1, 8, 64, 64)
BatchNorm            face_landmark_453                           (8,)       (1, 8, 64, 64)
ReLU                 face_landmark_455                             --       (1, 8, 64, 64)
Convolution          face_landmark_465                   (1, 1, 8, 4)       (1, 4, 64, 64)
BatchNorm            face_landmark_480                           (4,)       (1, 4, 64, 64)
Softmax              face_landmark_22982                           --         (1, 2, 1, 1)
IR network structure is saved as [0a5cedc744e94ec69e4f7adc95cbdc62.json].
IR network structure is saved as [0a5cedc744e94ec69e4f7adc95cbdc62.pb].
IR weights are saved as [0a5cedc744e94ec69e4f7adc95cbdc62.npy].
Parse file [0a5cedc744e94ec69e4f7adc95cbdc62.pb] with binary format successfully.
Target network code snippet is saved as [0a5cedc744e94ec69e4f7adc95cbdc62.py].
/usr/local/lib/python2.7/dist-packages/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Tensorflow file is saved as [./caffe_mv3_gp_small_s/saved_model.pb], generated by [0a5cedc744e94ec69e4f7adc95cbdc62.py] and [0a5cedc744e94ec69e4f7adc95cbdc62.npy].

ysh329 commented 4 years ago

TensorFlow模型后量化

https://tensorflow.google.cn/lite/performance/model_optimization?hl=zh_cn

工具选择

首先，检查 hosted models 中的模型是否适合您的应用程序。如果没有，我们建议用户从 post-training quantization tool 开始，因为它广泛适用的，且无需训练数据。

对于精度和延迟目标没有达到，或者需要硬件加速器支持情况， quantization-aware training 是更好的选择。参见 Tensorflow 模型优化工具包Tensorflow Model Optimization Toolkit 中的的其他优化技术。

ysh329 commented 4 years ago

上面是文档里的内容，咱们自定义的模型不在 hosts models 里，而且因为只是跑个int8的cpu性能，作为自定义模型走 post-training quantization tool 更合适。

https://tensorflow.google.cn/lite/performance/post_training_quantization?hl=zh_cn

Optimization Methods

There are several post-training quantization options to choose from. Here is a summary table of the choices and the benefits they provide:

Technique	Benefits	Hardware
Dynamic range quantization	4x smaller, 2-3x speedup	CPU
Full integer quantization	4x smaller, 3x+ speedup	CPU, Edge TPU, Microcontrollers
Float16 quantization	2x smaller, potential GPU acceleration	CPU, GPU

The following decision tree can help determine which post-training quantization method is best for your use case:


pip install tensorflow==1.8.0

ysh329 commented 4 years ago

// dynamic_range_quant.py

import tensorflow as tf

save_model_dir = "caffe_mv3_gp_small_s"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()

报错

$ python dynamic_range_quant.py
/usr/local/lib/python2.7/dist-packages/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Traceback (most recent call last):
  File "dynamic_range_quant.py", line 4, in <module>
    converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
AttributeError: 'module' object has no attribute 'lite'

ysh329 commented 4 years ago

pip install tensorflow==1.15.0

期间遇到报错 Cannot uninstall 'wrapt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

pip install -U --ignore-installed wrapt enum34 simplejson netaddr

然后重装tensorflow不行，全部删掉wrapt，

rm -rf /usr/local/lib/python2.7/dist-packages/wrapt*

然后pip install -U --ignore-installed wrapt enum34 simplejson netaddr

ysh329 commented 4 years ago

用miniconda重新搞了环境：https://docs.conda.io/en/latest/miniconda.html，下载linux的64位的miniconda，安装完成后`source ~/.bashrc`激活环境变量。

创建新环境：

conda create -n tensorflow python=2.7
conda activate tensorflow
pip install tensorflow==1.15

安装完tensorflow的环境后，再次执行python dynamic_range_quant.py，截取最后的日志报错：

RuntimeError: MetaGraphDef associated with tags set(['serve']) could not be found in SavedModel. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli`
available_tags: [set([u'train'])]

根据日志报错提示，修改代码如下：

// dynamic_range_quant.py

import tensorflow as tf

save_model_dir = "caffe_mv3_gp_small_s"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir, set_tag=set([u'train']))
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()

再次执行python dynamic_range_quant.py，日志如下：

$ python dynamic_range_quant.py
2020-06-11 10:49:48.546510: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: /usr/lib/x86_64-linux-gnu/libcuda.so.1: file too short; LD_LIBRARY_PATH: /usr/lib/x86_64-linux-gnu/:/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/:/usr/lib/x86_64-linux-gnu:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2020-06-11 10:49:48.546576: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2020-06-11 10:49:48.546611: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (yq01-guanghui-test144.yq01.baidu.com): /proc/driver/nvidia/version does not exist
2020-06-11 10:49:48.558742: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2500020000 Hz
2020-06-11 10:49:48.560872: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f963ec61500 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-06-11 10:49:48.560935: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /root/miniconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow_core/lite/python/convert_saved_model.py:60: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
2020-06-11 10:49:54.289508: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2020-06-11 10:49:54.289717: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2020-06-11 10:49:54.479264: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:786] Optimization results for grappler item: graph_to_optimize
2020-06-11 10:49:54.479613: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:788]   function_optimizer: function_optimizer did nothing. time = 0.006ms.
2020-06-11 10:49:54.479641: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:788]   function_optimizer: function_optimizer did nothing. time = 0ms.
WARNING:tensorflow:From /root/miniconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow_core/lite/python/util.py:249: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /root/miniconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow_core/python/framework/graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
2020-06-11 10:49:55.763828: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2020-06-11 10:49:55.764039: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2020-06-11 10:49:56.759698: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:786] Optimization results for grappler item: graph_to_optimize
2020-06-11 10:49:56.759880: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:788]   constant_folding: Graph size after: 3952 nodes (996), 5730 edges (-822), time = 596.037ms.
2020-06-11 10:49:56.759898: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:788]   constant_folding: Graph size after: 3952 nodes (0), 5730 edges (0), time = 247.555ms.

ysh329 commented 4 years ago

发现没有保存模型，而且原有模型业务改动，查了文档https://tensorflow.google.cn/api_docs/python/tf/compat/v1/lite/TFLiteConverter?hl=en，代码完善如下：

import tensorflow as tf

saved_model_dir = "caffe_mv3_gp_small_s"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir, tag_set=set([u'train']))
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()
open(saved_model_dir + ".tflite", "wb").write(tflite_quant_model)

ysh329 commented 4 years ago

失败，放弃

yuenshome / yuenshome.github.io

caffe模型TensorFlow Lite上线并量化 #120

Caffe模型先转TensorFlow

TensorFlow模型后量化