TensorFlow Lite Converter changed output_shapes

sherrylsin20 commented 2 years ago

1. System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
TensorFlow installation (pip package or built from source): pip
TensorFlow library (version, if pip package or github SHA, if built from source): 2.5.0 (for model training and exporting to saved model) & 2.7.0 (for exporting to TFLite in Google Colab)

2. Code

Provide code to help us reproduce your issues using one of the following options: Trained my model with model_main_tf2.py script before exporting to saved model using the exporter_main_v2.py script. After that I uploaded my saved model to Google Drive before trying to convert the model to TFLite format. Here is the code for TFLite Convert:

converter = tf.compat.v1.lite.TFLiteConverter.from_saved_model(MODEL_PATH,
    input_arrays=["serving_default_input_tensor"] ,
    input_shapes={'serving_default_input_tensor':[1,640,640,3]},
    output_arrays=['StatefulPartitionedCall:1', 'StatefulPartitionedCall:2', 'StatefulPartitionedCall:4', 'StatefulPartitionedCall:5'])
converter.allow_custom_ops=False
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
open("/content/new.tflite", "wb").write(tflite_model)

Reference for making the custom object detection model:

1) Reference Object Detection Tutorial: Installation until exporting to saved model

3. Failure after conversion

If the conversion is successful, but the generated model is wrong, then state what is wrong: It changed the output shapes of each outputs from this:

INFO:tensorflow:output tensors info:
INFO:tensorflow:Tensor's key in saved_model's tensor_map: num_detections
INFO:tensorflow: tensor name: StatefulPartitionedCall:5, shape: (1), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: detection_classes
INFO:tensorflow: tensor name: StatefulPartitionedCall:2, shape: (1, 100), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: detection_boxes
INFO:tensorflow: tensor name: StatefulPartitionedCall:1, shape: (1, 100, 4), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: raw_detection_scores
INFO:tensorflow: tensor name: StatefulPartitionedCall:7, shape: (1, 51150, 101), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: detection_multiclass_scores
INFO:tensorflow: tensor name: StatefulPartitionedCall:3, shape: (1, 100, 101), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: detection_anchor_indices
INFO:tensorflow: tensor name: StatefulPartitionedCall:0, shape: (1, 100), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: detection_scores
INFO:tensorflow: tensor name: StatefulPartitionedCall:4, shape: (1, 100), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: raw_detection_boxes
INFO:tensorflow: tensor name: StatefulPartitionedCall:6, shape: (1, 51150, 4), type: DT_FLOAT

to this:

print(input_details)
[{'name': 'serving_default_input_tensor', 'index': 0, 'shape': array([  1, 640, 640,   3], dtype=int32), 
'shape_signature': array([  1, 640, 640,   3], dtype=int32), 'dtype': <class 'numpy.uint8'>, 
'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 
'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]

print(output_details)
[{'name': 'StatefulPartitionedCall', 'index': 2449, 'shape': array([1, 1], dtype=int32), 'shape_signature': array([ 1, -1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:1', 'index': 2502, 'shape': array([1, 1, 1], dtype=int32), 'shape_signature': array([ 1, -1, -1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:2', 'index': 2484, 'shape': array([1, 1], dtype=int32), 'shape_signature': array([ 1, -1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:3', 'index': 2467, 'shape': array([1, 1, 1], dtype=int32), 'shape_signature': array([ 1, -1, -1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:4', 'index': 2431, 'shape': array([1, 1], dtype=int32), 'shape_signature': array([ 1, -1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:5', 'index': 2414, 'shape': array([1], dtype=int32), 'shape_signature': array([1], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:6', 'index': 519, 'shape': array([    1, 51150,     4], dtype=int32), 'shape_signature': array([    1, 51150,     4], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, 
{'name': 'StatefulPartitionedCall:7', 'index': 527, 'shape': array([    1, 51150,   101], dtype=int32), 'shape_signature': array([    1, 51150,   101], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]

As you can see, the output shapes of StatefulPartitionedCall 0 - 4 changed. Any solution? I've searched around but I haven't seen any problems similar with the problems I have.

4. Information

Pre trained models used: ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8

The only things I've changed from the original pipeline was these: num_classes: 100 (because I have 100 classes) batch_size: 2 deleted all augmentation steps (flip and crop) fine_tune_checkpoint="my_path/pre-trained-models/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/checkpoint/ckpt-0" num_steps = 100000 fine_tune_checkpoint_type= "detection" use_bfloat16= false train_input_reader=changed the path eval_input_reader=changed the path

I'm sorry if my explanation seems confusing, this is the second time I've ever opened an issue in Github. If you need the saved model and tflite, I'll be willing to share it. Please help!