search

Tencent / ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform
Other
20.13k stars 4.14k forks source link

Bad performance for int8 inference on XuanTie 906 (RISC-V) #5447

Open vassilijnadarajah opened 4 months ago

vassilijnadarajah commented 4 months ago

detail | 详细描述 | 詳細な説明

I was wondering if maybe anyone knows or might have an idea as to why the inference time on the XuanTie 906 (RISC-V) processor is so slow for int8 models?
The image can easily be reproduced by running the NCNN benchmark on the Alwinner D1 (1x XuanTie 906). The displayed numbers are the runtimes in milliseconds.

Screenshot 2024-05-06 at 16 01 11 Screenshot 2024-05-06 at 16 01 38

Thanks!

nihui commented 4 months ago

Currently ncnn lacks good optimization for int8 models using risc-v and rvv

You may be interested in contributions here https://github.com/plctlab/rvspoc-s2310-ncnn/pull/2