ValueError: Dimension size, given by scalar input 1 must be in range [-1, 1)

[SaneBow]

I am trying to convert these tflite models: https://github.com/breizhn/DTLN-aec/tree/main/pretrained_models The command I ran was: tflite2tensorflow --model_path dtln_aec_128_2.tflite --flatc_path ../flatc --schema_path ../schema.fbs --output_pb But ran into this error:

ValueError: Dimension size, given by scalar input 1 must be in range [-1, 1) for '{{node split_functional_5/lstm_6/lstm_cell_6/StatefulPartitionedCall/split}} = Split[T=DT_FLOAT, num_split=4](split_functional_5/lstm_6/lstm_cell_6/StatefulPartitionedCall/split/split_dim, BiasAdd_functional_5/lstm_6/lstm_cell_6/StatefulPartitionedCall/BiasAdd)' with input shapes: [], [512] and with computed input tensors: input[0] = <1>.

Tried with both pip installation and docker version. Is it a bug or something that not supported?

[PINTO0309]

Fixed a behavior bug with keep_num_dims in FullyConnected. v1.7.5 047b86b53c4897b1f1a383c1e74460d01511f6e7

[SaneBow]

Thanks for the quick fix. It can run and convert to pb now. But when I load the weight quantized model it faild:

Traceback (most recent call last):
  File "run_aec.py", line 224, in <module>
    process_folder(args.model, args.in_folder, args.out_folder)
  File "run_aec.py", line 210, in process_folder
    os.path.join(new_directories[idx], file_names[idx]),
  File "run_aec.py", line 115, in process_file
    interpreter_1.set_tensor(input_details_1[2]["index"], lpb_mag)
  File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 423, in set_tensor
    self._interpreter.SetTensor(tensor_index, value)
ValueError: Cannot set tensor: Dimension mismatch. Got 3 but expected 4 for input 2.

I see this during quantization:

WARNING:tensorflow:Issue encountered when serializing variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
to_proto not supported in EAGER mode.
WARNING:tensorflow:Issue encountered when serializing trainable_variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
to_proto not supported in EAGER mode.

This may be unrelated to this issue though.

Edit: I confirmed the error is not related to this issue here by verifying that --output_no_quant_float32_tflite works. Thanks for the fix. We can closing this issue.

[PINTO0309]

@SaneBow There are three different input values, but only one is 4-dimensional. I think it's simply that the number of "tensor_index" is swapped between the float32 and INT8 models.

[SaneBow]

@PINTO0309 Thanks for the reply and your amazing project . I am not familiar with tensorflow. What does this mean? Is it the model cannot be quantized or some other bugs in the tool?

[PINTO0309]

@SaneBow Rather than a bug in the tool, I am aware of the current specification of the tool, which assumes image processing. In other words, my tool basically expects [N, H, W, C] for the input shape. In addition, I know in advance that an error will occur if there are multiple shapes of input data.

However, when I think about quantizing these particular models, I also know that it is easy to quantize them by simply modifying one part of the tool.

• tflite2tensorflow.py

      def representative_dataset_gen():
          # if calib_ds_type == 'tfds':
          #     for data in raw_test_data.take(10):
          #         image = data['image'].numpy()
          #         images = []
          #         for shape in input_shapes:
          #             data = tf.image.resize(image, (shape[1], shape[2]))
          #             tmp_image = eval(string_formulas_for_normalization) # Default: (data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]
          #             tmp_image = tmp_image[np.newaxis,:,:,:]
          #             images.append(tmp_image)
          #         yield images
          # elif calib_ds_type == 'numpy':
          #     for idx in range(raw_test_data.shape[0]):
          #         image = raw_test_data[idx]
          #         images = []
          #         for shape in input_shapes:
          #             data = tf.image.resize(image, (shape[1], shape[2]))
          #             tmp_image = eval(string_formulas_for_normalization) # Default: (data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]
          #             tmp_image = tmp_image[np.newaxis,:,:,:]
          #             images.append(tmp_image)
          #         yield images
          calib_data_input3 = np.random.rand(1, 1, 257).astype(np.float32)
          calib_data_input4 = np.random.rand(1, 1, 257).astype(np.float32)
          calib_data_input5 = np.random.rand(1, 2, 128, 2).astype(np.float32)
          yield [calib_data_input5, calib_data_input3, calib_data_input4]

  TensorFlow's quantization method requires about 100 pieces of data that were used during training. This data provides the clues necessary for TensorFlow to automatically calculate the appropriate range of INT8 types to quantize based on the features in the training data. Strictly speaking, however, I know from experience that even if you don't use the data used during training, the impact will be minimal, so this time I passed a random number in Numpy.

• Integer Quantized tflite file - model_integer_quant.tflite https://drive.google.com/file/d/1cYWVoskz2vX9G3hBAiYtOzUbMyoRAWkG/view?usp=sharing

In short, technically any model can be quantized, but I have neglected to customize the tool. :crying_cat_face:

[SaneBow]

I see, so this tool expect image processing models but I tried to use it on audio processing models that's why I failed. Now I have only the tflite and the first step of converting back to saved model seems to produce incorrect model with dimension mismatch. If I can have correct saved model then weight quantization should be easy. Which part do I need to modify to make it work for a different input shape? Is it an easy task or will it involve a lot of changes?

[PINTO0309]

@SaneBow I took your issue seriously and understood that there is a demand for it, so I upgraded the version to support quantization of models with multiple input and multiple shape input. The current version is v1.8.1.

[SaneBow]

That's fast. You are amazing. Now I upgrade to v1.8.1 and try to convert those models again. But I got this error when load one of the converted model:

Traceback (most recent call last):
  File "run_aec.py", line 138, in stage2
    interpreter_2.set_tensor(input_details_2[0]["index"], estimated_block)
  File "/Users/xb/.pyenv/versions/DTLN/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 423, in set_tensor
    self._interpreter.SetTensor(tensor_index, value)
ValueError: Cannot set tensor: Dimension mismatch. Got 3 but expected 4 for input 0.

What I am trying to do is to quantize the models in https://github.com/breizhn/DTLN-aec I used the following command:

tflite2tensorflow --model_path dtln_aec_128_2.tflite --flatc_path ../flatc --schema_path ../schema.fbs --output_pb
tflite2tensorflow --model_path dtln_aec_128_2.tflite --flatc_path ../flatc --schema_path ../schema.fbs --output_weight_quant_tflite

Here this input estimated_block for dtln_aec_128_2.tflite is an output from the model dtln_aec_128_1.tflite. Please let me know if you need any help testing.

[PINTO0309]

I tried to implement the same workflow, but no error occurred.

tflite2tensorflow \
--model_path dtln_aec_128_2.tflite \
--flatc_path ../flatc \
--schema_path ../schema.fbs \
--output_pb

tflite2tensorflow \
--model_path dtln_aec_128_2.tflite \
--flatc_path ../flatc \
--schema_path ../schema.fbs \
--output_weight_quant_tflite

Once the .json file is generated, it will not be regenerated, so try deleting dtln_aec_128_2.json by hand and then try again.

[PINTO0309]

Ahh, you mean that you get an error at runtime.

[PINTO0309]

In the process of transforming the model, TensorFlow swaps the order of the input layers by itself.

From. run_aec.py

        interpreter_2.set_tensor(input_details_2[1]["index"], states_2) # float32[1,2,128,2]
        interpreter_2.set_tensor(input_details_2[0]["index"], estimated_block) # float32[1,1,257]
        interpreter_2.set_tensor(input_details_2[2]["index"], in_lpb) # float32[1,1,257]

To. run_aec.py

        interpreter_2.set_tensor(input_details_2[0]["index"], states_2)
        interpreter_2.set_tensor(input_details_2[1]["index"], estimated_block)
        interpreter_2.set_tensor(input_details_2[2]["index"], in_lpb)

Or

        interpreter_2.set_tensor(input_details_2[1]["index"], states_2)
        interpreter_2.set_tensor(input_details_2[2]["index"], estimated_block)
        interpreter_2.set_tensor(input_details_2[0]["index"], in_lpb)

Or

        interpreter_2.set_tensor(input_details_2[2]["index"], states_2)
        interpreter_2.set_tensor(input_details_2[1]["index"], estimated_block)
        interpreter_2.set_tensor(input_details_2[0]["index"], in_lpb)

[SaneBow]

That solved my problem. Thanks for helping a tensorflow noob out. One last question: Any way to convert the model so it aligns with the original one?

[PINTO0309]

After executing the first command, we need to rewrite the json file by replacing the order of the output layers.

[SaneBow]

That works. Thanks. Your projects are all very cool.