Xilinx / xup_aie_training

Hands-on experience programming AI Engines using Vitis Unified Software Platform
https://xilinx.github.io/xup_aie_training/
MIT License
36 stars 9 forks source link

[Issue]: Compilation Not Completing for more than 100 AIE Kernels Used #30

Closed soumitraghosh388 closed 1 month ago

soumitraghosh388 commented 1 month ago

Is there an existing issue related to your problem?

Lab

Vector Add 1

Section

While Compiling for target=hw

Step

Compilation step

Description

When I use target=hw, the compilation step is not completing (even after 12 hours) and is hanging at mapper phase of compilation. This issue is a generic issue which can be encountered for any lab experiment. I also tried to map kernels to tile locations and am using runtime_ratio=0.9 i.e, 1 kernel mapped to each tile.

The command for compilation is shown below :

aiecompiler -v --target=hw \
    --stacksize=12000 \
    --heapsize=10000 \
    --Xmapper=DisableFloorplanning \
    -include="{vitis_path}/aietools/include" \
    -include="{app_path}/aie/src"  \
    -include="{app_path}/aie/data" \
    --platform=/opt/xilinx/platforms/xilinx_vck5000_gen4x8_qdma_2_202220_1/xilinx_vck5000_gen4x8_qdma_2_202220_1.xpfm \
    {app_path}/aie/src/graph.cpp \
    -workdir=work 2>&1 | tee compile.log

But when I am using less than 100 kernels compilation and aiesimulation is working fine.

Instance

Cloud

Vitis

****** Xilinx Vitis Development Environment
****** Vitis v2022.2.2 (64-bit)
  **** SW Build 3716524 on 2023-02-17-15:43:19
    ** Copyright 1986-2022 Xilinx, Inc. All Rights Reserved.

XRT

Version              : 2.14.384
Branch               : 2022.2
Hash                 : 090bb050d570d2b668477c3bd0f979dc3a34b9db
Hash Date            : 2022-12-09 08:55:04
XOCL                 : unknown, unknown
XCLMGMT              : unknown, unknown

Platform

xilinx_vck5000_gen4x8_qdma_2_202220_1

Linux distribution

LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Rocky
Description:    Rocky Linux release 8.8 (Green Obsidian)
Release:    8.8
Codename:   GreenObsidian

Anything else?

No response

mariodruiz commented 1 month ago

Why have you change the heapsize and stacksize? Can you try removing this?

soumitraghosh388 commented 1 month ago

I have tried removing heapsize and stacksize but issue remains. I have also tried in hacc cluster node, issue persists.

mariodruiz commented 1 month ago

Is this for the vadd lab? how did you change the number of kernels?

soumitraghosh388 commented 1 month ago

No, this is not for vadd lab but for a generic scenario. I can change the no. of kernels by creating N no. of instances of it and using runtime_ratio = 1 for each of them in graph.hpp.

mariodruiz commented 1 month ago

Hi @soumitraghosh388, If the code you are running is not part of this repository, the issues does not belong here.

Please, post in the Xilinx forums for support, feel free to email me the issue thread if you do not receive a response in a reasonable time.

soumitraghosh388 commented 1 month ago

Hi Mario, I was wondering where to post this issue. Thanks for the suggestion.