search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.44k stars 624 forks source link

DPU-RNN for ZCU102/ZCU104 #396

Closed YacobHampelt closed 2 years ago

YacobHampelt commented 3 years ago

Hi,

I would like to use Xilinx DPU to accelerate RNNs/LSTMs on a ZynqMP Platform, preferably ZCU102 or ZCU104. As far as I know the DPU-TRD dosn't support those layers. But I have found DPU-RNN (https://github.com/Xilinx/Vitis-AI/tree/master/dsa/DPU-for-RNN), which supports RNNs. However, it appears to be only available for Alveo U25/U50. Is there any possibility to get it to work on other FPGA boards (like the aforementioned) as well? Is there any roadmap indicating when it might be supported? If not, can you please explain why only U25/U50 are supported (any limitations of ZCU102/ZCU104, why it wouldn't work)?

Best regards

beekil commented 3 years ago

Hi, currently DPU for RNN is available for Alveo U25/U50lv. We have some recent works to have RNN-T on the 7nm Versal platform but no plan on the MPSoC platform. Literally speaking a single batch RNN could also work on ZU+ platforms but we need to have detailed technical analysis. I hope this could answer your questions.

YacobHampelt commented 3 years ago

Hey,

thank you very much for your reply! Could you maybe give me a hint, what the limitation of the ZU+ platforms is, that makes it harder to run the DPU-RNN for them? Is it too little memory, for example?

Best regards

qianglin-xlnx commented 2 years ago

Hi @YacobHampelt Sorry for the late reply. Yes, the limited on-chip memory resource makes it difficult to directly run the xrnn kernel on ZCU102 or ZCU104. For example, our kernel on Alveo U25 takes 56 URAMs and 486.5 BRAMs, while ZCU102 has 912 BRAMs, ZCU104 has 312 BRAMs and 96 URAMs. Hardware resource optimization will be required.

julianop99 commented 1 year ago

Hi, I do have the same need - I want to run a light LSTM model on the Zynq US+. Is it ever available or not yet?

hetanshi221 commented 3 months ago

Hi, I do have the same need - I want to run a light LSTM model on the Zynq US+. Is it ever available or not yet?

Hi, I am also working on deploying LSTM on my ZCU104 board. Did it work for you?