search

AXERA-TECH / ax-samples

Samples code for world class Artificial Intelligence SoCs for computer vision applications.
BSD 3-Clause "New" or "Revised" License
209 stars 45 forks source link

Metrics for `.joint` models #76

Closed mikel-brostrom closed 1 year ago

mikel-brostrom commented 1 year ago

Could you provide metrics for the available exported models, or at least a subset of them, so that we can understand what performance degradation is to be expected when INT8-quantinzing models with pulsar?

mikel-brostrom commented 1 year ago

Moreover, no detailed information is available for the architecture topology of the exported models. How well does pulsar handle Swish, GELU activations when exporting and specially INT8 quantization?

mikel-brostrom commented 1 year ago

https://github.com/AXERA-TECH/ax-samples/issues/73

kalcohol commented 1 year ago

Hi mikel-brostrom, u can get pulsar from ax-pi wiki so compile your models and then have a better know of each performance & metrics of your models on ax620 chips. the released models are not aiming to show the performance & metrics, the samples are provided to showing that how to use the npu api.

so #73 that demo models cannot provide performance degradation of dataset as your wish. (we do not suggest customers using the models in final produt directly, so the models are not set as its best precision)