Closed fjzhangcr closed 2 years ago
@fjzhangcr Please perform the full integer quantization to be able the map the operations to edgeTPU.
Here is some sample example: https://github.com/google-coral/edgetpu/issues/586#issuecomment-1119842063
that works in int8 ,but my first purpose is to run in float16~~~ i notice there are several tips about model requirements here: https://coral.ai/docs/edgetpu/models-intro/#model-requirements the first one is about the quantilization~~~ i want to make it sure : EDGETPU ONLY SUPPORT 8BIT-QUANTILIZATION, do not support float16 or float32~~ am i right?
============================ Model requirements If you want to build a TensorFlow model that takes full advantage of the Edge TPU for accelerated inferencing, the model must meet these basic requirements:
Tensor parameters are quantized (8-bit fixed-point numbers; int8 or uint8). Tensor sizes are constant at compile-time (no dynamic sizes). Model parameters (such as bias tensors) are constant at compile-time. Tensors are either 1-, 2-, or 3-dimensional. If a tensor has more than 3 dimensions, then only the 3 innermost dimensions may have a size greater than 1. The model uses only the operations supported by the Edge TPU (see table 1 below).
the first one is about the quantilization~~~ i want to make it sure : EDGETPU ONLY SUPPORT 8BIT-QUANTILIZATION, do not support float16 or float32~~ am i right?
Yes, you are right.
thank you~~~
Description
i have yolo model converted to tflite already, but edgetpu_compiler said all the operations can not run on edgetpu....so, i make a model just contain a conv2d layer... the edgetpu_compiler said such a simple model tflite model not run on edgetpu... i am confuse~~help me
my tf=2.8, edgetpu runtime=14, edgetpu_compiler=16
Click to expand!
### Issue Type Bug ### Operating System Windows 10 ### Coral Device Dev Board ### Other Devices _No response_ ### Programming Language Python 3.7 ### Relevant Log Output ```shell the tiny demo model contains just one conv2d is below: # -------------------------------------- SAVED_MODEL_DIR="just_conv2d" tflite_model_filename='just_conv2d.tflite' input_layer = tf.keras.layers.Input([512, 512, 3]) y=tf.keras.layers.Conv2D( filters=6,kernel_size=3,strides=1,padding='same')(input_layer) model=tf.keras.Model(inputs=input_layer,outputs=y) model.summary() model.save(SAVED_MODEL_DIR) Model: "model_5" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_5 (InputLayer) [(None, 512, 512, 3)] 0 conv2d_223 (Conv2D) (None, 512, 512, 6) 168 ================================================================= Total params: 168 Trainable params: 168 Non-trainable params: 0 then i conver it to a tflite file: #----------------------------------- converter = tf.lite.TFLiteConverter.from_saved_model(SAVED_MODEL_DIR) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.float16] converter.target_spec.supported_ops = [ tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS] converter.experimental_new_converter = True tflite_model = converter.convert() with open(tflite_model_filename, 'wb') as f: f.write(tflite_model) then i run the python interpreter.invoke() to make sure it runs well on PC #----------------------------------- interpreter = tf.lite.Interpreter(model_path=tflite_model_filename) interpreter.allocate_tensors() input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() # Explore the model input output shape. print(input_details) print(output_details) input_shape = input_details[0]['shape'] print("input_shape is ",input_shape) for i, output_detail in enumerate(output_details): output_shape = output_detail['shape'] print("No {} output_shape is {}".format(i,output_shape)) # Test the model on random input data. input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) interpreter.set_tensor(input_details[0]['index'], input_data) print("feeding model with input_shape is ",input_data.shape) interpreter.invoke() # The function `get_tensor()` returns a copy of the tensor data. # Use `tensor()` in order to get a pointer to the tensor. for i in range(len(output_details)): output_data_tmp = interpreter.get_tensor(output_details[i]['index']) output_shape_tmp = output_data_tmp.shape print("inference done! No {} output_shape is {}".format(i,output_shape_tmp)) the output are: [{'name': 'serving_default_input_5:0', 'index': 0, 'shape': array([ 1, 512, 512, 3]), 'shape_signature': array([ -1, 512, 512, 3]), 'dtype':