a model with just one conv2d layer can not run on Edge TPU~~

[fjzhangcr]

Description

i have yolo model converted to tflite already, but edgetpu_compiler said all the operations can not run on edgetpu....so, i make a model just contain a conv2d layer... the edgetpu_compiler said such a simple model tflite model not run on edgetpu... i am confuse~~help me

my tf=2.8, edgetpu runtime=14, edgetpu_compiler=16

Click to expand!

### Issue Type Bug ### Operating System Windows 10 ### Coral Device Dev Board ### Other Devices _No response_ ### Programming Language Python 3.7 ### Relevant Log Output ```shell the tiny demo model contains just one conv2d is below: # -------------------------------------- SAVED_MODEL_DIR="just_conv2d" tflite_model_filename='just_conv2d.tflite' input_layer = tf.keras.layers.Input([512, 512, 3]) y=tf.keras.layers.Conv2D( filters=6,kernel_size=3,strides=1,padding='same')(input_layer) model=tf.keras.Model(inputs=input_layer,outputs=y) model.summary() model.save(SAVED_MODEL_DIR) Model: "model_5" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_5 (InputLayer) [(None, 512, 512, 3)] 0 conv2d_223 (Conv2D) (None, 512, 512, 6) 168 ================================================================= Total params: 168 Trainable params: 168 Non-trainable params: 0 then i conver it to a tflite file: #----------------------------------- converter = tf.lite.TFLiteConverter.from_saved_model(SAVED_MODEL_DIR) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.float16] converter.target_spec.supported_ops = [ tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS] converter.experimental_new_converter = True tflite_model = converter.convert() with open(tflite_model_filename, 'wb') as f: f.write(tflite_model) then i run the python interpreter.invoke() to make sure it runs well on PC #----------------------------------- interpreter = tf.lite.Interpreter(model_path=tflite_model_filename) interpreter.allocate_tensors() input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() # Explore the model input output shape. print(input_details) print(output_details) input_shape = input_details[0]['shape'] print("input_shape is ",input_shape) for i, output_detail in enumerate(output_details): output_shape = output_detail['shape'] print("No {} output_shape is {}".format(i,output_shape)) # Test the model on random input data. input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) interpreter.set_tensor(input_details[0]['index'], input_data) print("feeding model with input_shape is ",input_data.shape) interpreter.invoke() # The function `get_tensor()` returns a copy of the tensor data. # Use `tensor()` in order to get a pointer to the tensor. for i in range(len(output_details)): output_data_tmp = interpreter.get_tensor(output_details[i]['index']) output_shape_tmp = output_data_tmp.shape print("inference done! No {} output_shape is {}".format(i,output_shape_tmp)) the output are: [{'name': 'serving_default_input_5:0', 'index': 0, 'shape': array([ 1, 512, 512, 3]), 'shape_signature': array([ -1, 512, 512, 3]), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}] [{'name': 'StatefulPartitionedCall:0', 'index': 3, 'shape': array([ 1, 512, 512, 6]), 'shape_signature': array([ -1, 512, 512, 6]), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}] input_shape is [ 1 512 512 3] No 0 output_shape is [ 1 512 512 6] then i switch to ubuntu to use edgetup_compiler to compile it for edgetpu: indeed@indeed-virtual-machine:~/Desktop/tflite$ edgetpu_compiler just_conv2d.tflite -s Edge TPU Compiler version 16.0.384591198 Started a compilation timeout timer of 180 seconds. Model compiled successfully in 1 ms. Input model: just_conv2d.tflite Input size: 1.70KiB Output model: just_conv2d_edgetpu.tflite Output size: 1.26KiB On-chip memory used for caching model parameters: 0.00B On-chip memory remaining for caching model parameters: 0.00B Off-chip memory used for streaming uncached model parameters: 0.00B Number of Edge TPU subgraphs: 0 Total number of operations: 3 Operation log: just_conv2d_edgetpu.log Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs. Number of operations that will run on Edge TPU: 0 Number of operations that will run on CPU: 3 Operator Count Status CONV_2D 1 Operation is working on an unsupported data type DEQUANTIZE 1 Tensor has unsupported rank (up to 3 innermost dimensions mapped) DEQUANTIZE 1 Operation is working on an unsupported data type Compilation child process completed within timeout period. Compilation succeeded! ```

[hjonnala]

@fjzhangcr Please perform the full integer quantization to be able the map the operations to edgeTPU.

Here is some sample example: https://github.com/google-coral/edgetpu/issues/586#issuecomment-1119842063

[fjzhangcr]

that works in int8 ,but my first purpose is to run in float16~~~ i notice there are several tips about model requirements here: https://coral.ai/docs/edgetpu/models-intro/#model-requirements the first one is about the quantilization~~~ i want to make it sure : EDGETPU ONLY SUPPORT 8BIT-QUANTILIZATION, do not support float16 or float32~~ am i right?

============================ Model requirements If you want to build a TensorFlow model that takes full advantage of the Edge TPU for accelerated inferencing, the model must meet these basic requirements:

Tensor parameters are quantized (8-bit fixed-point numbers; int8 or uint8). Tensor sizes are constant at compile-time (no dynamic sizes). Model parameters (such as bias tensors) are constant at compile-time. Tensors are either 1-, 2-, or 3-dimensional. If a tensor has more than 3 dimensions, then only the 3 innermost dimensions may have a size greater than 1. The model uses only the operations supported by the Edge TPU (see table 1 below).

[hjonnala]

the first one is about the quantilization~~~ i want to make it sure : EDGETPU ONLY SUPPORT 8BIT-QUANTILIZATION, do not support float16 or float32~~ am i right?

Yes, you are right.

[fjzhangcr]

thank you~~~

[google-coral-bot[bot]]

Are you satisfied with the resolution of your issue? Yes No