Optimizations for Pi AI hat (Hailo)

vitoplantamura / OnnxStream

Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK.

https://yolo.vitoplantamura.com/

Other

1.86k stars 84 forks source link

Optimizations for Pi AI hat (Hailo) #77

Open eiddor opened 4 months ago

eiddor commented 4 months ago

I'm taking a crack at trying to integrate OnnxStream and OnnxStreamGUI (web) into the same Docker container (while also trying to learn PHP in the process) on my RPi 5 which also happens to have the new Hailo AI Hat that was released a little while ago.

Assuming I can pass the Hailo device into the container, I'm wondering if OnnxStream can use the hat easily or if that would require code changes.

Thanks!

vitoplantamura commented 4 months ago

hi,

unfortunately OnnxStream does not support the Hailo AI Hat natively :-(

Vito