search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.44k stars 624 forks source link

Can not run the compiled xmodel (pytorch) in inference phase on KV260 (B4096 & B3136) #1411

Open rezajahadi opened 6 months ago

rezajahadi commented 6 months ago

Hello,

I have quantized a resnet18 model using the resnet18_quant.py script provided by Vitis AI in the "example/vai_quantizer/pytorch" folder. The xmodel was successfully created for the quantized model. I used this command to run the script:

python resnet18_quant.py --quant_mode test --model_dir pre_trained_models/ --data_dir tiny-imagenet-200/ --target DPUCZDX8G_ISA1_B3136 --batch_size=1 --deploy --subset_len 1

Then I compile the xmodel by this command:

vai_c_xir -x ResNet_int.xmodel -a /opt/vitis_ai/compiler/arch/DPUCZDX8G/KV260/arch.json -o customized_xmodel_pytorch/ -n resnet18

This command gives me the final xmodel file for DPU implementation. However, when I run DPU using this xmodel on my target device (KV260), I face this error:

"" Kernel Restarting The kernel appears to have died. It will restart automatically. """

KrishnaGaihre commented 5 months ago

Which DPU image are you running on KV260? Have you checked "Vitis AI runtime example on that board" for sanity check if only the boot image is created by you?

rezajahadi commented 5 months ago

At the time of writing the issue, I couldn't have the inference on KV260. Right after that, I could run the xmodel on KV260. It was because the Vitis ai libraries were not installed on Ubuntu OS, that was running on Kria KV260. I got the last update of Vitis AI image from Xilinx website and flashed the SD card. The problem was solved.

But I face another problem. Once I want to use the models from pre trained weights, using the command "model.load_state_dict(torch.load(pretrained_weights_path))", the model cant be quantized. The error is that the model is not traceable. Could you please help me on this matter?

On Fri, Mar 15, 2024 at 8:48 AM Krishna Gaihre @.***> wrote:

Which DPU image are you running on KV260? Have you checked "Vitis AI runtime example https://github.com/Xilinx/Vitis-AI/tree/3.0/examples/vai_runtime/resnet50 on that board" for sanity check if only the boot image is created by you?

— Reply to this email directly, view it on GitHub https://github.com/Xilinx/Vitis-AI/issues/1411#issuecomment-1999596010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV2DGZXW2U3SJZYT5VIMY43YYLUZVAVCNFSM6AAAAABEMGKFVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJZGU4TMMBRGA . You are receiving this because you authored the thread.Message ID: @.***>

-- Reza Jahadi MSc. Electrical and Computer Engineering Department of Electrical Engineering Lakehead University, Canada