pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch
https://pytorch.org/executorch/
Other
1.65k stars 276 forks source link

How to perform inference and gathering accuracy metrics on executorch model #2397

Open mmingo848 opened 5 months ago

mmingo848 commented 5 months ago

Hi, I am having trouble finding solid documentation that explains how to do the following with executorch (stable):

I have applied quantization and other optimizations to the original model and exported it to .pte. I'd like to see the accuracy after these techniques were applied. I followed the following tutorial for exporting the model. If we can't do the above items on the directly exported .pte file, then is there a way we can based on the below steps for preparing the model for edge dialect?

https://pytorch.org/executorch/stable/tutorials/export-to-executorch-tutorial.html

mikekgfb commented 3 months ago

@mmingo848 We'll be publishing a solution to this - hopefully very soon!