search

quic / qidk

Other
76 stars 16 forks source link

Int4 quantization #23

Closed Piotr94 closed 2 months ago

Piotr94 commented 2 months ago

Hi!

According to the documentation some platforms (for example Snapdragon 8 gen 2) enable using int4 computations. I would like to ask if it is possible to use them through SNPE? If not can you tell me how can I implement int4 quantization on qualcomm platforms?

Kind regards Piotr

quic-rneti commented 2 months ago

Yes - INT4 quantization is possible with SNPE on Snapdragon 8 Gen2, and Snapdragon 8 Gen3. Please follow SDK documentation, and let us know for any specific questions.