google-coral / pycoral

Python API for ML inferencing and transfer-learning on Coral devices
https://coral.ai
Apache License 2.0
340 stars 138 forks source link

Segmentation Models label size mismatch in output tensors #114

Open stakach opened 1 year ago

stakach commented 1 year ago

Description

Downloading the U-Net MobileNet v2 segmentation models from the coral.ai website they seem to have an output tensor that matches the input tensor. i.e. input: 1x256x256x3, output: 1x256x256x3

However all the examples expect there to be an output per-label per-pixel. i.e. input: 1x256x256x3, output: 1x256x256x21

and looking at tutorials on the web from a few years ago, they all seem to match the example code and the models they are using all work.

So my question is, how does one interpret the output of these segmentation models?

Click to expand! ### Issue Type Documentation Bug ### Operating System _No response_ ### Coral Device _No response_ ### Other Devices _No response_ ### Programming Language _No response_ ### Relevant Log Output _No response_
hjonnala commented 1 year ago

matches the input tensor. i.e. input: 1x256x256x3, output: 1x256x256x3

However all the examples expect there to be an output per-label per-pixel. i.e. input: 1x256x256x3, output: 1x256x256x21

Coral models are traind to classify 3 classes (1x256x256x3) and other models might have trained to classify 21 classes(1x256x256x21).

There can be argamax applied on the final layer for some models. If not argmax would be applied in post processing. image

stakach commented 1 year ago

I can use the argmax models without issue, however the labels file on the website includes many more than 3 classes https://raw.githubusercontent.com/google-coral/test_data/master/pet_labels.txt