MLIR-based AI Engine toolchain

GitHub Pull Requests

This repository contains an MLIR-based toolchain for AI Engine-enabled devices, such as AMD Ryzen™ AI and Versal™. This repository can be used to generate low-level configurations for the AI Engine portion of these devices. AI Engines are organized as a spatial array of tiles, where each tile contains AI Engine cores and/or memories. The spatial array is connected by stream switches that can be configured to route data between AI Engine tiles scheduled by their programmable Data Movement Accelerators (DMAs). This repository contains MLIR representations, with multiple levels of abstraction, to target AI Engine devices. This enables compilers and developers to program AI Engine cores, as well as describe data movements and array connectivity. A Python API is made available as a convenient interface for generating MLIR design descriptions. Backend code generation is also included, targeting the aie-rt library. This toolchain uses the AI Engine compiler tool which is part of the AMD Vitis™ software installation: these tools require a free license for use from the Product Licensing Site.

This project is primarily intended to support the open-source community, particularly tool builders, with low-level access to AIE devices and enable the development of a wide variety of programming models from higher level abstractions. We provide an example programming flow: Interface Representation for hands-ON (IRON) close-to-metal programming of the AIE-array. IRON is an open access toolkit enabling performance engineers to build fast and efficient, often specialized designs through a set of Python language bindings around the mlir-aie dialect. As such, it contains some examples, however this project is not intended to represent an end-to-end compilation flow for all application designs. If you're looking for an out-of-the-box experience for highly efficient machine learning, check out the AMD Ryzen™ AI Software Platform.

Getting Started for AMD Ryzen™ AI - Linux Quick Setup Instructions

These instructions will guide you through everything required for building and executing a program on the Ryzen™ AI NPU, starting from a fresh bare-bones Ubuntu 24.10 install with Linux 6.11 kernel.

Initial Setup

Update BIOS:

Be sure you have the latest BIOS for your laptop or mini PC, this will ensure the NPU (sometimes referred to as IPU) is enabled in the system. You may need to manually enable the NPU: Advanced → CPU Configuration → IPU

NOTE: Some manufacturers only provide Windows executables to update the BIOS, please do this before installing Ubuntu.

BIOS Settings:

Turn off SecureBoot (Allows for unsigned drivers to be installed): BIOS → Security → Secure boot → Disable

Prerequisites

Install AIETools

Supporting AMD Ryzen™ AI with AMD XDNA™/AIE-ML (AIE2) and AMD XDNA™ 2 (AIE2P): Install AMD Vitis™ AIE Essentials

Install Vitis™ AIE Essentials from Ryzen AI Software 1.3 Early Accesss. We will assume you use the installation directory, /tools/ryzen_ai-1.3.0/vitis_aie_essentials.

This is an early access lounge, you must register and be granted access at this time.
1. Download VAIML Installer for Linux based compilation: ryzen_ai-1.3.0ea1.tgz
2. Extract the required tools:
```
  tar -xzvf ryzen_ai-1.3.0ea1.tgz
  cd ryzen_ai-1.3.0
  mkdir vitis_aie_essentials
  mv vitis_aie_essentials*.whl vitis_aie_essentials
  cd vitis_aie_essentials
  unzip vitis_aie_essentials*.whl
```
Set up an AI Engine license.
1. Get a local license for AI Engine tools from https://www.xilinx.com/getlicense.
2. Copy your license file (Xilinx.lic) to your preferred location, e.g. /opt/Xilinx.lic:

Setup your environment using the following script for Vitis™ for AIETools:

#!/bin/bash
#################################################################################
# Setup Vitis AIE Essentials
#################################################################################
export AIETOOLS_ROOT=/tools/ryzen_ai-1.3.0/vitis_aie_essentials
export PATH=$PATH:${AIETOOLS_ROOT}/bin
export LM_LICENSE_FILE=/opt/Xilinx.lic

Install the XDNA™ Driver

Install the following prerequisite packages.
```
sudo apt install \
libidn11-dev
```
Clone the XDNA™ driver repository and its submodules.
```
git clone https://github.com/amd/xdna-driver.git
export XDNA_SRC_DIR=$(realpath xdna-driver)
cd xdna-driver
git reset --hard 3d5a8cf1af2adfbb6306ad71b45e5f3e1ffc5b37
git submodule update --init --recursive
```
The submodules use SSH remotes. You will need a GitHub account and locally installed SSH keys to pull the submodules. Follow these instructions to set up an SSH key. Alternatively, edit .gitmodules to use HTTPS instead of SSH.

Install XRT. (Below steps are adapted from here.)

Install XRT prerequisites.

cd $XDNA_SRC_DIR
sudo ./tools/amdxdna_deps.sh

Build XRT. Remember to source the aietools/Vitis setup script from above.
```
cd $XDNA_SRC_DIR/xrt/build
./build.sh -noert -noalveo
```

Install XRT.

cd $XDNA_SRC_DIR/xrt/build/Release
sudo apt reinstall ./xrt_202420.2.18.0_24.10-amd64-xrt.deb ./xrt_202420.2.18.0_24.10-amd64-xbflash.deb

An error is expected in this step. Ignore it.

Build XDNA-Driver. Below steps are adapted from here.

cd $XDNA_SRC_DIR/build
./build.sh -release
./build.sh -package

Install XDNA™.

cd $XDNA_SRC_DIR/build/Release
sudo apt reinstall ./xrt_plugin.2.18.0_ubuntu24.10-x86_64-amdxdna.deb

Check that the NPU is working if the device appears with xrt-smi:

source /opt/xilinx/xrt/setup.sh
xrt-smi examine

At the bottom of the output you should see:

 Devices present
 BDF             :  Name             
------------------------------------
 [0000:66:00.1]  :  RyzenAI-npu1

Install IRON and MLIR-AIE Prerequisites

Install the following packages needed for MLIR-AIE:

sudo apt install \
build-essential clang clang-14 lld lld-14 cmake python3-venv python3-pip libxrender1 libxtst6 libxi6 virtualenv

Install g++13 and opencv needed for some programming examples:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update
sudo apt install gcc-13 g++-13 -y
sudo apt install libopencv-dev python3-opencv

Remember to source the Vitis™ AIE Essentials setup script from above.
Remember to source the XRT setup script: source /opt/xilinx/xrt/setup.sh

Install IRON for AMD Ryzen™ AI AIE Application Development

Clone the mlir-aie repository, best under /home/username for speed (yourPathToBuildMLIR-AIE):
```
git clone https://github.com/Xilinx/mlir-aie.git
cd mlir-aie
```
Source utils/quick_setup.sh to setup the prerequisites and install the mlir-aie and llvm compiler tools from whls.

Build an IRON Design for AIEs in the AMD Ryzen™ AI NPU

Remember to set up your environment including Vitis™ AIE Essentials, your license, XRT, and IRON
  source yourVitisSetupScript.sh
  export LM_LICENSE_FILE=/opt/Xilinx.lic
  source /opt/xilinx/xrt/setup.sh
  source utils/env_setup.sh my_install/mlir_aie my_install/mlir my_install/llvm-aie

For your design of interest, for instance from programming_examples, 2 steps are needed: (i) build the AIE design and then (ii) build the host code.

Build Device AIE Part

Goto the design of interest and run make

Build and Run Host Part

Build: Goto the same design of interest folder where the AIE design just was built (see above)
```
make <testName>.exe
```
Note that the host code target has a .exe file extension even on Linux. Although unusual, this is an easy way for us to distinguish whether we want to compile device code or host code.
Run (program arguments are just an example for vector_scalar_add design)
```
make run
```