MaxPool2d Unsupported on DPUCZDX8G_ISA1_B4096

[manudwd]

Hi I was running the model inspected on a simplistic tensorflow model on Vitis 2.5.0,

[VAI INFO] Update include_bias_corr: False
[VAI INFO] Update include_fast_ft: False
[VAI INFO] Update include_cle: False
[VAI WARNING] Layer max_pooling2d_46(MaxPooling2D)'s `MaxPooling2D` is not supported by target
[VAI WARNING] Layer max_pooling2d_47(MaxPooling2D)'s `MaxPooling2D` is not supported by target
[VAI WARNING] Layer max_pooling2d_48(MaxPooling2D)'s `MaxPooling2D` is not supported by target
[VAI INFO] Inspect Results:
[MODEL INFO]:
________________________________________________________________________________________________________________________
Model Name: model_3
________________________________________________________________________________________________________________________
ID          Name                    Type                    Device      Notes                                           
========================================================================================================================
0/9         input_4                 InputLayer              INPUT                                                       
------------------------------------------------------------------------------------------------------------------------
1/9         conv2d_629              Conv2D<relu>            DPU                                                         
------------------------------------------------------------------------------------------------------------------------
2/9         max_pooling2d_46        MaxPooling2D            CPU         `MaxPooling2D` is not supported by target       
------------------------------------------------------------------------------------------------------------------------
3/9         conv2d_630              Conv2D<relu>            DPU                                                         
------------------------------------------------------------------------------------------------------------------------
4/9         max_pooling2d_47        MaxPooling2D            CPU         `MaxPooling2D` is not supported by target       
------------------------------------------------------------------------------------------------------------------------
5/9         conv2d_631              Conv2D<relu>            DPU                                                         
------------------------------------------------------------------------------------------------------------------------
6/9         max_pooling2d_48        MaxPooling2D            CPU         `MaxPooling2D` is not supported by target       
------------------------------------------------------------------------------------------------------------------------
7/9         flatten_8               Flatten                 DPU                                                         
------------------------------------------------------------------------------------------------------------------------
8/9         dense_27                Dense<relu>             DPU                                                         
------------------------------------------------------------------------------------------------------------------------
9/9         dense_28                Dense<linear>           DPU                                                         
------------------------------------------------------------------------------------------------------------------------
========================================================================================================================
[SUMMARY INFO]:
- [Target Name]: DPUCZDX8G_ISA1_B4096
- [Target Type]: DPUCZDX8G
- [Total Layers]: 10
- [Layer Types]: InputLayer(1) Conv2D<relu>(3) MaxPooling2D(3) Flatten(1) Dense<relu>(1) Dense<linear>(1) 
- [Partition Results]: INPUT(1) DPU(6) CPU(3) 
========================================================================================================================
[NOTES INFO]:
- [2/9] Layer max_pooling2d_46 (Type:MaxPooling2D, Device:CPU):
    * `MaxPooling2D` is not supported by target
- [4/9] Layer max_pooling2d_47 (Type:MaxPooling2D, Device:CPU):
    * `MaxPooling2D` is not supported by target
- [6/9] Layer max_pooling2d_48 (Type:MaxPooling2D, Device:CPU):
    * `MaxPooling2D` is not supported by target
========================================================================================================================
[VAI INFO] Start plotting model to model.svg
[VAI INFO] Inspected model has been plotted in: model.svg.
[VAI INFO] Start dumping inspected results to inspect_results.txt
[VAI INFO] Inspected results has been dumped in: inspect_results.txt.
[VAI INFO] Inspect Finished.

When I realized that MaxPooling2D is unsupported, upon checking the https://docs.xilinx.com/r/en-US/ug1414-vitis-ai/Currently-Supported-Operators max-pool is mentioned but I will presume its only for 1D, I'd really appreciate either someone correcting me on MaxPool2D existence and usage for vitis-ai, or creating a pull-request to implement it.

[zhenzhen-AMD]

Hi @manudwd ， Could you please provide the complete code for us to better assist you?

[manudwd]

@zhenzhen-AMD Thanks the help. The original model is described in the following manner. DPU Target: DPUCZDX8G_ISA1_B4096

Vitis Verison: 2.5.0 (CPU)

TensorFlow Version: 2.8.0 I am using functional method to describe the model.

def LeNet():
    input_layer = layers.Input(shape=(250, 90, 1))

    x = layers.Conv2D(32, (7, 7), strides=(3, 1), activation='relu')(input_layer)
    x = layers.MaxPooling2D((2, 2))(x)
    x = layers.Conv2D(64, (5, 4), strides=(2, 2), padding="same", activation='relu')(x)
    x = layers.MaxPooling2D((2, 2))(x)
    x = layers.Conv2D(96, (3, 3), strides=(1, 1), activation='relu')(x)
    x = layers.MaxPooling2D((2, 2))(x)

    x = layers.Flatten()(x)
    x = layers.Dense(128, activation='relu')(x)
    output_layer = layers.Dense(7)(x)

    model = Model(inputs=input_layer, outputs=output_layer)
    return model

## Training
model.save( 'UT_HAR_tf.h5', save_format='h5')

Followed by loading and inspecting the model, which is when the warning in the original post is raised.

    model = tf.keras.models.load_model('Res18_RFSoC/UT_HAR_tf.h5')
    optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)
    loss_fn tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    model.compile(optimizer=optimizer,
              loss=loss_fn,
              metrics=['accuracy'])
    model.build((None, 1, 250, 90))

    WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.

inspector.inspect_model(model, 
                        plot=True, 
                        plot_file="model.svg", 
                        dump_results=True, 
                        dump_results_file="inspect_results.txt", 
                        verbose=0)

[zhenzhen-AMD]

Hi @manudwd ,

Thank you for raising this issue. I have successfully replicated the problem you encountered. The issue has been addressed in a subsequent release. I recommend using the latest version, 3.5. For reference, here's the code I used to replicate the issue:

import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras import layers
from tensorflow_model_optimization.quantization.keras import vitis_inspect

def LeNet():
    input_layer = layers.Input(shape=(250, 90, 1))

    x = layers.Conv2D(32, (7, 7), strides=(3, 1), activation='relu')(input_layer)
    x = layers.MaxPooling2D((2, 2))(x)
    x = layers.Conv2D(64, (5, 4), strides=(2, 2), padding="same", activation='relu')(x)
    x = layers.MaxPooling2D((2, 2))(x)
    x = layers.Conv2D(96, (3, 3), strides=(1, 1), activation='relu')(x)

    x = layers.MaxPooling2D((2, 2))(x)
    x = layers.Flatten()(x)
    x = layers.Dense(128, activation='relu')(x)
    output_layer = layers.Dense(7)(x)

    model = Model(inputs=input_layer, outputs=output_layer)
    return model

model = LeNet()

## Training
model.save( 'UT_HAR_tf.h5', save_format='h5')

model = tf.keras.models.load_model('./UT_HAR_tf.h5')
optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)
loss_fn=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=optimizer,
          loss=loss_fn,
          metrics=['accuracy'])

#model.build((None, 1, 250, 90))

inspector = vitis_inspect.VitisInspector(target='DPUCZDX8G_ISA1_B4096')
inspector.inspect_model(model, 
                        plot=True, 
                        plot_file="model.svg", 
                        dump_results=True, 
                        dump_results_file="inspect_results.txt", 
                        verbose=0)

[manudwd]

@zhenzhen-AMD That makes sense! Thanks for the help, before I close this issue, would there be a method for me to port this layer to DPU on Vitis AI v2.5? I am working with PYNQ DPU, which supports Vitis AI v2.5 as of now.

[zhenzhen-AMD]

Hi @manudwd ， The results from inspect are for reference only. The issue arises from the incomplete inspect functionality of vai_q_tensorflow2 in VAI 2.5. Deployment to the DPU is determined by the xmodelproduced after final quantization and compilation. Thus, if you intend to deploy MaxPool2d to DPU, you can use VAI 2.5 for quantization and compilation.

[manudwd]

@zhenzhen-AMD edit: I realized you're saying that the inspector functionality isn't complete, aka I would still be running on DPU with VAI2.5 post quantization and deployment. One final thing, is there a way for me to verify that?

[quentonh]

@manudwd I believe that what @zhenzhen-AMD is saying is that the Model Inspector report for Vitis AI 2.5 is not valid. You see, that was the very earliest version of the Inspector. In subsequent releases, the Model Inspector actually executes the compiler in the background to determine with a higher degree of accuracy whether or not the model can be deployed. In Vitis AI 2.5, the Inspector did not do that, and didn't actually account for the DPU architecture that was targeted.

[manudwd]

@quentonh Yeah took me a second but I got that! I guess my only question now is whether there is a way where I can check whether all my layers have indeed been moved to the DPU since inspector won't help me with that on VAI v2.5.