search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.49k stars 630 forks source link

Vitis-AI support for other video sources #880

Closed Werterror closed 2 years ago

Werterror commented 2 years ago

Hi all, I looked through the datasheets but I couldn't find any mentions about video sources other then files stored on SD card and USB cameras. What about HDMI/Ethernet ports on ZCU102 and Kria KV260? And J7 & J8 - IAS connectors, J9 - Raspberry Pi Camera Connector just on Kria KV260, are cameras connected to those ports are also plug and play like USB cameras, are they supported in any way? BR

tianfang-fafafa commented 2 years ago

hi @Werterror , please look into this https://docs.xilinx.com/v/u/en-US/ug1182-zcu102-eval-bd; It may help you.

Werterror commented 2 years ago

hi @tianfang-fafafa, since Vitis-AI is launched with PetaLinux I assumed that after instaling drivers for a device (other then USB webcam) I simply could select device when running an example with different command.

tianfang-fafafa commented 2 years ago

hi @Werterror ， do you have any other questions? you can provide steps and err log, please

Werterror commented 2 years ago

hi @tianfang-fafafa, I would like to know if my approach is correct since it's not described, or at least I couldn't find it, and I am not very experienced with any distro of linux. I will soon try to connect thermal camera via ethernet and usb and then I can provide logs.

tianfang-fafafa commented 2 years ago

hi @Werterror ， As far as I know, usb camera can be used on zcu102 board, but zcu102 board just supports vitis-ai example running with image and video files stored on SD card.

Werterror commented 2 years ago

Hi @tianfang-fafafa, with image provided for ZCU102 you can use USB camera, moreover vitis-ai examples can be run with camera input. I already have tested most of them. Now I would like to see if I can use thermal camera instead. Right now I don't have thermal camera that could be connected to the board via USB but I got couple that could be connected by the Ethernet. And I was wondering how syntax for thermal camera connected via the Ethernet or HDMI to the board would look like, or if its even possible.

tianfang-fafafa commented 2 years ago

@Werterror ， you are right. USB camera can be used as input for vitis-ai example. image but have no test for thermal camera on zcu102 board. Please develop it follow the linux driver development process.