モデルを量子化する

kondounagi commented 4 years ago

$ $ freeze_graph --help
usage: freeze_graph [-h] [--input_graph INPUT_GRAPH]
                    [--input_saver INPUT_SAVER]
                    [--input_checkpoint INPUT_CHECKPOINT]
                    [--checkpoint_version CHECKPOINT_VERSION]
                    [--output_graph OUTPUT_GRAPH]
                    [--input_binary [INPUT_BINARY]]
                    [--output_node_names OUTPUT_NODE_NAMES]
                    [--restore_op_name RESTORE_OP_NAME]
                    [--filename_tensor_name FILENAME_TENSOR_NAME]
                    [--clear_devices [CLEAR_DEVICES]]
                    [--initializer_nodes INITIALIZER_NODES]
                    [--variable_names_whitelist VARIABLE_NAMES_WHITELIST]
                    [--variable_names_blacklist VARIABLE_NAMES_BLACKLIST]
                    [--input_meta_graph INPUT_META_GRAPH]
                    [--input_saved_model_dir INPUT_SAVED_MODEL_DIR]
                    [--saved_model_tags SAVED_MODEL_TAGS]

optional arguments:
  -h, --help            show this help message and exit
  --input_graph INPUT_GRAPH
                        TensorFlow 'GraphDef' file to load.
  --input_saver INPUT_SAVER
                        TensorFlow saver file to load.
  --input_checkpoint INPUT_CHECKPOINT
                        TensorFlow variables file to load.
  --checkpoint_version CHECKPOINT_VERSION
                        Tensorflow variable file format
  --output_graph OUTPUT_GRAPH
                        Output 'GraphDef' file name.
  --input_binary [INPUT_BINARY]
                        Whether the input files are in binary format.
  --output_node_names OUTPUT_NODE_NAMES
                        The name of the output nodes, comma separated.
  --restore_op_name RESTORE_OP_NAME
                        The name of the master restore operator. Deprecated,
                        unused by updated loading code.
  --filename_tensor_name FILENAME_TENSOR_NAME
                        The name of the tensor holding the save path.
                        Deprecated, unused by updated loading code.
  --clear_devices [CLEAR_DEVICES]
                        Whether to remove device specifications.
  --initializer_nodes INITIALIZER_NODES
                        Comma separated list of initializer nodes to run
                        before freezing.
  --variable_names_whitelist VARIABLE_NAMES_WHITELIST
                        Comma separated list of variables to convert to
                        constants. If specified, only those variables will be
                        converted to constants.
  --variable_names_blacklist VARIABLE_NAMES_BLACKLIST
                        Comma separated list of variables to skip converting
                        to constants.
  --input_meta_graph INPUT_META_GRAPH
                        TensorFlow 'MetaGraphDef' file to load.
  --input_saved_model_dir INPUT_SAVED_MODEL_DIR
                        Path to the dir with TensorFlow 'SavedModel' file and
                        variables.
  --saved_model_tags SAVED_MODEL_TAGS
                        Group of tag(s) of the MetaGraphDef to load, in string
                        format, separated by ','. For tag-set contains
                        multiple tags, all tags must be passed in.

$ $ vai_q_tensorflow --help
usage:
    usage: vai_q_tensorflow command [Options]

    examples:
      show help       : vai_q_tensorflow --help
      quantize a model: vai_q_tensorflow quantize --input_frozen_graph frozen_graph.pb --input_nodes xxx --output_nodes yyy --input_shapes zzz --input_fn module.calib_input
      inspect a model : vai_q_tensorflow inspect --input_frozen_graph frozen_graph.pb
      dump quantized model : vai_q_tensorflow dump --input_frozen_graph quantize_results/quantize_eval_model.pb --input_fn module.dump_input

Xilinx's Quantization Tools Vai_q_tensorflow v1.0.0 Build for Tensorflow
1.12.0

positional arguments:
  {quantize,inspect,dump}
                        Specify a command for vai_q_tensorflow

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --input_frozen_graph INPUT_FROZEN_GRAPH
                        The path to input frozen graph(.pb) (default: )
  --output_dir OUTPUT_DIR
                        The directory to save the quantization results
                        (default: ./quantize_results)
  --weight_bit WEIGHT_BIT
                        The target bit width for weights/biases (default: 8)
  --activation_bit ACTIVATION_BIT
                        The target bit width for activation (default: 8)
  --method {0,1}        The method for quantization, options are: 0: non-
                        overflow method, make sure no values are saturated
                        during quantization, may get bad results incase of
                        outliers. 1: min-diffs method, allow saturation for
                        large values during quantization to get smaller
                        quantization errors. This method is slower than method
                        0 but has higher endurance to outliers. (default: 1)
  --calib_iter CALIB_ITER
                        The iterations of calibration, total number of images
                        for calibration = calib_iter * batch_size (default:
                        100)
  --input_nodes INPUT_NODES
                        The name list of input nodes of the subgraph to be
                        quantized, comma separated. Used together with
                        output_nodes. When generating the model for deploy,
                        only the subgraph between input_nodes and output_nodes
                        will be included. Please set it to the begining of the
                        main body fo the model to quantize, such as the nodes
                        after data preprocessing and augmentation. (default: )
  --input_shapes INPUT_SHAPES
                        The shape list of input_nodes, The shape must be a
                        4-dimension shape for each node, comma separated, e.g.
                        1,224,224,3; Unknown size for batchsize is supported,
                        e.g. ?,224,224,3; In case of multiple input_nodes,
                        please assign the shape list of each node, separated
                        by `:`. e.g. ?,224,224,3:?,300,300,1 (default: )
  --output_nodes OUTPUT_NODES
                        The name list of output nodes of the subgraph to be
                        quantized, comma separated. Used together with
                        input_nodes. When generating the model for deploy,
                        only the subgraph between input_nodes and output_nodes
                        will be included. Please set it to the end of the main
                        body of the model to quantize, such as the nodes
                        before postprocessing. (default: )
  --ignore_nodes IGNORE_NODES
                        The name list of nodes to be ignored during
                        quantization, comma separated. The ignored nodes will
                        be left unquantized during quantization even if it is
                        quantizable. This argument has no effect for non-
                        quantizable nodes. (default: )
  --skip_check {0,1}    Set to 1 to skip the check for float model. (default:
                        0)
  --align_concat {0,1,2}
                        The strategy for alignment of the input quantize
                        positions for concat nodes. Set to 0 to align all
                        concat nodes, 1 to align the output concat nodes, 2 to
                        disable alignment (default: 0)
  --simulate_dpu {0,1}  Set to 1 to enable simulation of DPU. The behavior of
                        DPU for some operations are different from tensorflow.
                        For example, the dividing in LeakyRelu and AvgPooling
                        are replaced by bit-shifting, so there maybe slight
                        difference between DPU outputs and CPU/GPU outputs.
                        This quantizer will simulate the behavior for these
                        operations if this flag is set to 1 (default: 1)
  --input_fn INPUT_FN   The python importable function that provides the input
                        data. The format is `module_name.input_fn_name`, e.g.
                        'my_input_fn.input_fn'. The input_fn should take a
                        `int` object as input indicating the calibration step,
                        and should return a dict`(placeholder_node_name :
                        numpy.Array)` object for each call, which will be fed
                        into the model's placeholder nodes. (default: )
  --max_dump_batches MAX_DUMP_BATCHES
                        The maximum batches to be dumped (default: 1)
  --dump_float {0,1}    Set to 1 to dump the float weights/biases and
                        activation tensors together with the quantized
                        tensors. (default: 0)
  --gpu GPU             The gpu id used for quantization, comma separated.
                        (default: 0)
  --gpu_memory_fraction GPU_MEMORY_FRACTION
                        The gpu memory fraction used for quantization, between
                        0-1. (default: 0.5)

kondounagi commented 4 years ago

AYMの物理マシンにsshする，このとき窓際のルータに有線接続して，プライベートネットに入っておくこと

$ ssh -F /dev/null user@192.168.11.4                                                                                                                                                           Fri

kondounagi commented 4 years ago

byobuを立てて，model_developmentに移動，dockerに入る

$ byobu
$ cd /home/user/ai-edge/ai-edge-contest-4th/3rd/platform/Vitis-AI
$ ./gpu_docker_run.sh

kondounagi commented 4 years ago

dockerに入り，仮想環境をアクティベートする．
```
$ conda activate vitis-ai-tensorflow
```

kondounagi commented 4 years ago

このissue先頭の二つのコマンドが使える

kondounagi commented 4 years ago

以下で作業する

$ /workspace/model_development

SuzukiDaichi-git commented 3 years ago

vai_q_tensorflow quantize --input_frozen_graph unko/bisenetv2_from_onnx.pb --input_nodes image_array --input_shapes ?,1216,1936,3 --output_nodes category --input_fn input_fn.calib_input --method 1 --calib_iter 20 --output_dir ./Vitis-AI-Quantizer/vai_q_tensorflow/quantize_results

を実行すると, 以下のエラーが返ってきた.

Traceback (most recent call last):
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/bin/vai_q_tensorflow", line 11, in <module>
    sys.exit(run_main())
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 943, in run_main
    app.run(main=my_main, argv=[sys.argv[0]] + unparsed)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/absl/app.py", line 303, in run
    _run_main(main, args)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 942, in <lambda>
    my_main = lambda unused_args: main(unused_args, FLAGS)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 621, in main
    flags.skip_check)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 343, in quantize_frozen
    check_float_graph(input_graph_def, input_fn, q_config, s_config)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 258, in check_float_graph
    feed_dict = gen_feed_dict(input_tensors, inputs)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/utils.py", line 140, in gen_feed_dict
    "key {} not found, please check your input_fn.".format(name))
ValueError: key image_array not found, please check your input_fn.

SuzukiDaichi-git commented 3 years ago

input_fn.py内のcalib_inputで, return {"image_array": images} としたらこのエラーは出なくなった.

SuzukiDaichi-git commented 3 years ago

次のエラー

Traceback (most recent call last):
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/bin/vai_q_tensorflow", line 11, in <module>
    sys.exit(run_main())
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 943, in run_main
    app.run(main=my_main, argv=[sys.argv[0]] + unparsed)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/absl/app.py", line 303, in run
    _run_main(main, args)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 942, in <lambda>
    my_main = lambda unused_args: main(unused_args, FLAGS)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 621, in main
    flags.skip_check)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 343, in quantize_frozen
    check_float_graph(input_graph_def, input_fn, q_config, s_config)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/contrib/decent_q/python/decent_q.py", line 259, in check_float_graph
    sess.run(output_tensors, feed_dict)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run
    run_metadata_ptr)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1156, in _run
    (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 1216, 1936, 3) for Tensor 'image_array:0', which has shape '(1, 3, 1216, 1936)'

SuzukiDaichi-git commented 3 years ago

pytorchはNCHWの形でデータを扱うけど, tensorflowはNHWCの順で扱うらしく, 変換の過程ではこれをきちんと扱えていないようで, Quntizationする途中, shape関連のエラーが出ているよう.

参考 https://github.com/paulbauriegel/tensorflow-tools