search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.43k stars 621 forks source link

Error to load the Xmodel on the board #1073

Open Afef00 opened 1 year ago

Afef00 commented 1 year ago

I ran Vitis AI CPU docker container from host using ./docker_run.sh xilinx/vitis-ai-cpu:1.3.411. After successfully training my custom model using the following code

import os
import numpy as np
import tensorflow as tf
from sklearn.metrics import confusion_matrix
image_ordering_dim="tf"
def preprocess(X_train, X_test):
    """
    Convert from float64 to float32 and normalize normalize to decibels
    relative to full scale (dBFS) for the 4 sec clip.
    """
    X_train = X_train.astype('float32')
    X_test = X_test.astype('float32')

    X_train = np.array([(X - X.min()) / (X.max() - X.min()) for X in X_train])
    X_test = np.array([(X - X.min()) / (X.max() - X.min()) for X in X_test])
    return X_train, X_test
def prep_train_test(X_train, y_train, X_test, y_test, nb_classes):
    """
    Prep samples ands labels for Keras input by noramalzing and converting
    labels to a categorical representation.
    """
    print('Train on {} samples, validate on {}'.format(X_train.shape[0],
                                                       X_test.shape[0]))

    # normalize to dBfS
    X_train, X_test = preprocess(X_train, X_test)

    # Convert class vectors to binary class matrices
    Y_train = tf.keras.utils.to_categorical(y_train, nb_classes)
    Y_test = tf.keras.utils.to_categorical(y_test, nb_classes)

    return X_train, X_test, Y_train, Y_test
def keras_img_prep(X_train, X_test, img_dep, img_rows, img_cols):

    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)
    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

    return X_train, X_test, input_shape

def cnn(X_train, y_train, X_test, y_test, batch_size,
        nb_classes, epochs, input_shape):
    """

    """

    inputs = tf.keras.Input(shape=input_shape)
    x = tf.keras.layers.Conv2D(128, (3,3), activation='relu', input_shape=(input_shape),padding='same')(inputs)
    x = tf.keras.layers.MaxPooling2D((4,3),strides=(1, 3))(x)
    x = tf.keras.layers.Conv2D(32, (1,3), activation='relu',input_shape=(input_shape),padding='same')(x)
    x = tf.keras.layers.MaxPooling2D((1,3), strides=(1, 3))(x)
    x = tf.keras.layers.Flatten()(x)
    x = tf.keras.layers.Dense(512, activation='relu')(x)
    x = tf.keras.layers.Dense(512, activation='relu')(x)
    x = tf.keras.layers.Dropout(0.2)(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)

    model = tf.keras.Model(inputs=inputs, outputs=outputs, name='classifier')

    model.compile(loss='categorical_crossentropy',
                  optimizer='adadelta',
                  metrics=['accuracy'])

    history = model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs,
                        verbose=1, validation_data=(X_test, y_test))

    # Evaluate accuracy on test and train sets
    score_train = model.evaluate(X_train, y_train, verbose=0)
    print('Train accuracy:', score_train[1])
    score_test = model.evaluate(X_test, y_test, verbose=0)
    print('Test accuracy:', score_test[1])
    model.summary()

    return model, history

    X_train = np.load('train_samples.npz')
    y_train = np.load('train_labels.npz')
    X_test = np.load('test_samples.npz')
    y_test = np.load('test_labels.npz')

    X_train, y_train, X_test, y_test = \
        X_train['arr_0'], y_train['arr_0'], X_test['arr_0'], y_test['arr_0']

    batch_size = 32
    nb_classes = 2
    epochs = 7

    X_train, X_test, y_train, y_test = prep_train_test(X_train, y_train,
                                                       X_test, y_test,
                                                       nb_classes=nb_classes)

    # 513x125x1 for spectrogram with crop size of 125 pixels
    img_rows, img_cols, img_depth = X_train.shape[1], X_train.shape[2], 1

    X_train, X_test, input_shape = keras_img_prep(X_train, X_test, img_depth,
                                                  img_rows, img_cols)
    print('Fitting model...')
    model, history = cnn(X_train, y_train, X_test, y_test, batch_size,
                         nb_classes, epochs, input_shape)

After the quantization of the model I got this warning

!vai_q_tensorflow quantize \
    --input_frozen_graph frozen_graph_depression_test.pb \
    --input_fn input_func.calib_input \
    --output_dir quantized_depression \
    --input_nodes input_1 \
    --output_nodes dense_2/Softmax \
    --input_shapes ?,513,125,1\
    --calib_iter 32

INFO: Checking Float Graph... 2022-10-26 12:23:31.339312: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 434503680 exceeds 10% of system memory. 2022-10-26 12:23:31.479567: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 262656000 exceeds 10% of system memory. 2022-10-26 12:23:31.577909: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 85647360 exceeds 10% of system memory. 2022-10-26 12:23:31.608412: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 85647360 exceeds 10% of system memory. INFO: Float Graph Check Done. INFO: Calibrating for 32 iterations... N/A% (0 of 32) | | Elapsed Time: 0:00:00 ETA: --:--:--2022-10-26 12:23:42.735741: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 434503680 exceeds 10% of system memory. 100% (32 of 32) |########################| Elapsed Time: 0:02:33 Time: 0:02:33 INFO: Calibration Done. 2022-10-26 12:26:17.641835: W tensorflow/contrib/decent_q/utils/graph_quantizer.cc:1687] [DECENT_WARNING] Shift bias of node conv2d/Conv2D is 2147483638. It exceed range [-7, 16], modify quantize pos from 19 to 2147483641 2022-10-26 12:26:17.641863: W tensorflow/contrib/decent_q/utils/graph_quantizer.cc:1687] [DECENT_WARNING] Shift bias of node conv2d_1/Conv2D is 2147483637. It exceed range [-7, 16], modify quantize pos from 20 to 2147483641 2022-10-26 12:26:17.641867: W tensorflow/contrib/decent_q/utils/graph_quantizer.cc:1687] [DECENT_WARNING] Shift bias of node dense/MatMul is 2147483641. It exceed range [-2, 16], modify quantize pos from 21 to 2147483646 2022-10-26 12:26:17.641871: W tensorflow/contrib/decent_q/utils/graph_quantizer.cc:1687] [DECENT_WARNING] Shift bias of node dense_1/MatMul is 2147483638. It exceed range [-6, 16], modify quantize pos from 20 to 2147483642 2022-10-26 12:26:17.641873: W tensorflow/contrib/decent_q/utils/graph_quantizer.cc:1687] [DECENT_WARNING] Shift bias of node dense_2/MatMul is 2147483632. It exceed range [-6, 16], modify quantize pos from 26 to 2147483642 INFO: Generating Deploy Model... INFO: Deploy Model Generated. Quantization Summary
INFO: Output:
quantize_eval_model: quantized_depression/quantize_eval_model.pb
deploy_model: quantized_depression/deploy_model.pb

And the compilation went without any errors and I get the Xmodel.

However wen trying to load the Xmodel on the ZCU104 board as follow


from time import time
# import sklearn
import numpy as np
import cv2
# from sklearn import svm
import matplotlib.pyplot as plt
%matplotlib inline
from six.moves import urllib
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)
from pynq_dpu import DpuOverlay
overlay = DpuOverlay("dpu.bit")
overlay.load_model("depression_test.xmodel")

AssertionError Traceback (most recent call last)

in 1 from pynq_dpu import DpuOverlay 2 overlay = DpuOverlay("dpu.bit") ----> 3 overlay.load_model("depression_test.xmodel") 4 # overlay.load_model("depression_flatten_model.xmodel") /usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq_dpu/dpu.py in load_model(self, model) 169 self.graph = xir.Graph.deserialize(abs_model) 170 subgraphs = get_child_subgraph_dpu(self.graph) --> 171 assert len(subgraphs) == 1 172 self.runner = vart.Runner.create_runner(subgraphs[0], "run") AssertionError: I don't get what's the problem
qianglin-xlnx commented 1 year ago

Hi @Afef00 Do you use PYNQ system image? If not, you can refer to https://github.com/Xilinx/Vitis-AI/blob/1.3.1/demo/VART/inception_v1_mt_py/inception_v1.py python example.

Afef00 commented 1 year ago

@qianglin-xlnx Yes, I used PYNQ-DPU (PYNQ image 2.7). The problem is when I tried other models, such as those provided on Jupyter notebook or other Xmodels that I tested before, it works fine. I don't get what the problem is.

qianglin-xlnx commented 1 year ago

From the log you provide, the model you compiled has 2 DPU subgraphs. [UNILOG][INFO] Total device subgraph number 5, DPU subgraph number 2

While your program assert at the number of subgraph, which the DPU subgraph must be 1 in your program. Maybe you can modify this in your code.

--> 171 assert len(subgraphs) == 1

Afef00 commented 1 year ago

@qianglin-xlnx How can I modify this in the code?

Afef00 commented 1 year ago

I tried to modify the model architecture as follow:

  inputs = tf.keras.Input(shape=input_shape)
    x = tf.keras.layers.Conv2D(128, (3,3), activation='relu', input_shape=(input_shape))(inputs)
    x = tf.keras.layers.MaxPooling2D((4,3))(x)
    x = tf.keras.layers.Conv2D(32, (1,3), activation='relu')(x)
    x = tf.keras.layers.MaxPooling2D((1,3))(x)
    x = tf.keras.layers.Flatten()(x)
    x = tf.keras.layers.Dense(512, activation='relu')(x)
    x = tf.keras.layers.Dense(512, activation='relu')(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)

    model = tf.keras.Model(inputs=inputs, outputs=outputs, name='classifier')

However, when I tried to compile the model I got this error:

qianglin-xlnx commented 1 year ago

Hi @Afef00 Sorry about the late reply. Is this issue solved? My team does not have PYNQ-DPU. If you still have problem, you can ask PYNQ-DPU team for help.

Afef00 commented 1 year ago

No, actually this issue is not solved yet.