search

TexasInstruments / edgeai-benchmark

This repository has been moved. The new location is in https://github.com/TexasInstruments/edgeai-tensorlab
https://github.com/TexasInstruments/edgeai
Other
3 stars 0 forks source link

The compiled custom model shows wrong labels #5

Closed min0628 closed 2 years ago

min0628 commented 2 years ago

Hi, TI

I tried to using our custom ONNX model that trained by custom dataset. I compiled the model using tutorials/tutorial_detection.ipynb and it works on SK-TDA4VM. The result of EdgeAI Apps shows wrong labels. Our custom model has 3 classes, but EdgeAI Apps result has over the 3 classes. For example our model has vehicle class. but EdgeAI Apps result's label show vehicle/truck, vehicle/car and vehicle/motorcycle. I think model compile process has calibration using COCO dataset cause this problem.

Q1. Does calibration processing change compiled model number of classes? Q2. If I changed calibration_dataset='' in pipeline_configs, model's artifact doesn't create. Is this normal? Q3. In my situation, I can't skip calibration process so do i need to add our custom dataset in benchmark?

mathmanu commented 2 years ago

You need to you the appropriate calibration dataset - but this will primarily affect only the accuracy and not the classes themselves. The EdgeAI Apps may be written for a specific scenario, for example COCO in this case. I am guessing that you need to modify the EdgeAI Apps for your scenario.

min0628 commented 2 years ago

Opps. It was my mistake. I selected wrong model to compile. The new model has been compiled and works well. And modified EdgeAI Apps's classname, the result shows correct labels. Thanks.