onnx / models

A collection of pre-trained, state-of-the-art models in the ONNX format
http://onnx.ai/models/
Apache License 2.0
7.9k stars 1.4k forks source link

Resnet50, SqueezeNet and VGG19 represented twice from different source frameworks #82

Open crahrig opened 6 years ago

crahrig commented 6 years ago

The model zoo currently contains three models (Resnet50, SqueezeNet and VGG19) that each have two variants which is confusing to end consumers.

Ideally these should be de-duplicated. If that doesn't make sense then they should state their differences outside of origin framework and be organized in a way that places them all in the same sub-folder/path on the repo.

Following are pointers to the existing duplicated models:

lupesko commented 6 years ago

Thanks for the note @crahrig ! Fair point. Labeled and you are welcomed to submit a PR to merge into models listed here where there is more detailed documentation.

idilsulo commented 5 years ago

For the case of SqueezeNet models, there are not only two different models, but also one of the models results in poor accuracy values when I execute it.

For the first SqueezeNet model the link is given as the following: https://github.com/onnx/models/tree/master/models/image_classification/squeezenet

The probabilities that I get after executing this model is given below:

class=n02123045 tabby, tabby cat ; probability=0.411851
class=n02124075 Egyptian cat ; probability=0.295258
class=n02123159 tiger cat ; probability=0.251817
class=n02127052 lynx, catamount ; probability=0.037665
class=n02114367 timber wolf, grey wolf, gray wolf, Canis lupus ; probability=0.001155

Whereas, for the second model, the probabilities predicted for each class is pretty inaccurate. The link to this model is given as the following: https://github.com/onnx/models/tree/master/squeezenet

The probabilities are also given below:

class=n15075141 toilet tissue, toilet paper, bathroom tissue ; probability=1.000000
class=n02319095 sea urchin ; probability=1.000000
class=n02395406 hog, pig, grunter, squealer, Sus scrofa ; probability=1.000000
class=n02391049 zebra ; probability=1.000000
class=n02389026 sorrel ; probability=1.000000
snnn commented 5 years ago

Right now, I think, the top priority thing is getting them correct. All models here should conform to ONNX standard, and have a reasonable accuracy. I don't think it's a good time point to dedup models. Let's close issues like: #90 first.

idilsulo commented 5 years ago

@snnn What I mentioned above is not related to models being duplicate but rather one model having a really low accuracy value (not being correct) compared to the other one.