Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras format (NHWC). The purpose of this tool is to solve the massive Transpose extrapolation problem in onnx-tensorflow (onnx-tf). I don't need a Star, but give me a pull request.
Using converter for full int8 quantization using custom_input_op_name_np_data_path,
when
np_data = [["images", tmp_file, [[[[0, 0, 0]]]], [[[[255, 255, 255]]]]]]
This is the line in def representative_dataset_gen() function
normalized_calib_data = (calib_data[idx] - mean) / std
dividing by std generates float64 type and tf.convertor expects float32.
PINTO0309
commented
8 months ago
Using converter for full int8 quantization using custom_input_op_name_np_data_path, when np_data = [["images", tmp_file, [[[[0, 0, 0]]]], [[[[255, 255, 255]]]]]]
This is the line in def representative_dataset_gen() function normalized_calib_data = (calib_data[idx] - mean) / std
dividing by std generates float64 type and tf.convertor expects float32.
Simple fix:
normalized_calib_data = (calib_data[idx] - mean) / std yield_data_dict[model_input_name] = normalized_calib_data.astype('float32')