Vitis AI 3.0: Inspect show that DepthwiseConv2D + BatchNormalization + LeakyReLU not supported for DPU (DPUCZDX8G_ISA1_B4096)

[omgitsmj24]

I'm running into a problem trying to modify là custom model with similar traits to Yolov3 to match to DPU implementation using API_0 (or 1 and 2, not API_3), however, it seems that Vitis AI 3.0 can't match the block DepthwiseConv2D + BatchNormalization + LeakyReLU to DPU. But if I replace LeakyReLU with ReLU, it will work.

In UG1414, I've noticed this line in page 132:

For DPUCZDX8G, the activation LeakyReLU for depthwise-conv like operators is not enabled by default. About how to enable this activation, please refer to DPUCZDX8G for Zynq UltraScale+ MPSoCs (PG338)

Does this mean anything since I have already confirmed with my team that this feature have already been turned on in ZCU102?

Here below is a simple model with the block DW + BN + LeakyReLU not being used by DPU.

from keras.models import Model
from keras.layers import Input, Dense  # explicitly import Input layer
from tensorflow.keras import regularizers
from tensorflow_model_optimization.quantization.keras import vitis_inspect

l2_value = 0.000009999999747378752

input_layer = Input((960, 960, 3), name='input')
net = keras.layers.ZeroPadding2D(padding = ((0,1), (0,1)))(input_layer)
net = keras.layers.Conv2D(kernel_size=(3,3), strides = (2,2), 
                          filters=32, use_bias = False, 
                          kernel_regularizer = regularizers.L2(l2_value),
                          bias_regularizer = regularizers.L2(l2_value))(net)
net = keras.layers.BatchNormalization()(net)
net = keras.layers.LeakyReLU(alpha = 0.1015625)(net)
net = keras.layers.DepthwiseConv2D(kernel_size=(3, 3), padding='same', use_bias = False, strides = (1,1),
                                   bias_regularizer = regularizers.L2(l2_value))(net)
net = keras.layers.BatchNormalization()(net)
net = keras.layers.LeakyReLU(alpha = 0.1015625)(net)
prediction = keras.layers.Conv2D(filters=64, kernel_size=(1,1), use_bias = False,
                          kernel_regularizer = regularizers.L2(l2_value),
                          bias_regularizer = regularizers.L2(l2_value))(net) 

model_test = Model(inputs=input_layer, outputs=prediction)

model_test.summary()

inspector = vitis_inspect.VitisInspector(target='DPUCZDX8G_ISA1_B4096')
inspector.inspect_model(model_test,
                        plot=False,
                        dump_results=False,
                        verbose=2,
                        )

[VAI INFO] Inspect Results:
[MODEL INFO]:
________________________________________________________________________________________________________________________
Model Name: model_24
________________________________________________________________________________________________________________________
ID          Name                    Type                    Device      Notes                                           
========================================================================================================================
0/8         input                   InputLayer              INPUT                                                       
------------------------------------------------------------------------------------------------------------------------
1/8         zero_padding2d_7        ZeroPadding2D           DPU                                                         
------------------------------------------------------------------------------------------------------------------------
2/8         conv2d_11               Conv2D<linear>          DPU                                                         
------------------------------------------------------------------------------------------------------------------------
3/8         batch_normalization_16  BatchNormalization      DPU         Folded into previous layer conv2d_11            
------------------------------------------------------------------------------------------------------------------------
4/8         leaky_re_lu_6           LeakyReLU               DPU         Converted alpha 0.1 to 26./256. to match DPU    
                                                                        implementation                                  
------------------------------------------------------------------------------------------------------------------------
5/8         depthwise_conv2d_8      DepthwiseConv2D<linear> CPU                                                         
------------------------------------------------------------------------------------------------------------------------
6/8         batch_normalization_17  BatchNormalization      CPU         Folded into previous layer depthwise_conv2d_8   
------------------------------------------------------------------------------------------------------------------------
7/8         leaky_re_lu_7           LeakyReLU               CPU         Converted alpha 0.1 to 26./256. to match DPU    
                                                                        implementation; xir::Op{name =                  
                                                                        quant_depthwise_conv2d_8, type = depthwise-     
                                                                        conv2d-fix} has been assigned to CPU: [DPU does 
                                                                        not support activation type: LEAKYRELU.]        
------------------------------------------------------------------------------------------------------------------------
8/8         conv2d_12               Conv2D<linear>          DPU                                                         
------------------------------------------------------------------------------------------------------------------------
========================================================================================================================
[SUMMARY INFO]:
- [Target Name]: DPUCZDX8G_ISA1_B4096
- [Total Layers]: 9
- [Layer Types]: InputLayer(1) ZeroPadding2D(1) Conv2D<linear>(2) BatchNormalization(2) LeakyReLU(2) DepthwiseConv2D<linear>(1) 
- [Partition Results]: INPUT(1) DPU(5) CPU(3) 

Vitis AI 3.0 DPUCZDX8G_ISA1_B4096 docker image: vitis-ai-tensorflow2-cpu:latest

[JasonXijieJia]

Hi @omgitsmj24 The config of DPUCZDX8G_ISA1_B4096 doesn't support depthwise-conv2d-fix + activation LeakyReLU by default.
Please try to enable ALU_LEAKYRELU feature according to VivadoFlow or VitisFlow, and re-generate bitfiles, re-compile xmodel with the new arch.json file. You are welcome to feedback. Thank you

[qianglin-xlnx]

Closing since no activity for more than 2 months, please reopen if you still have any questions, thanks