Adding More Models to Modelzoo

[colah]

I'd really like get more models into lucid modelzoo.

Background

It seems like there are two major reservoirs of pre-trained models: tf slim and caffe. Most pre-trained models online come from one of those. In many cases, there are two different pre-trained models going by the same name floating around, because both projects re-implemented the model. (tf-hub seems to mostly be repackaging a subset of the tf-slim models.)

Typically, neither of these sources is the canonical model.

Questions / Considerations

Should we really be creating another model zoo, or is there a way we could just pull from an existing one?

• Only tf-hub seems provide models in an easy to import way; it's mostly a subset of slim.
• We want models from different sources
• We may want more meta-data than others

How do we deal with multiple different implementations of the "same" model?

• Post-fix names (eg. InceptionV3_slim ?)

It would be really nice to have the same model trained on multiple datasets for some experiments

• https://github.com/CSAILVision/places365

What meta-data should we be including about models?

[colah]

Initial work on importing models (eg. AlexNet and InceptionV3) in #85.

[colah]

Workflows for importing models

Slim -> Frozen

I've checked in some scripts that seem to work in #86 (lucid/scratch/scripts/import_slim/).

Caffe -> Frozen (caffe-tensorflow)

One solution is caffe-tensorflow which now has a --standalone-output-path for directly crating frozen models (see ethereon/caffe-tensorflow#76).

Getting this to work reliably takes a number of tricks, including using an old tensorflow version, running a caffe model update script, and manually cleaning up temp files.

I've checked in some scripts that seem to work in #86 (lucid/scratch/scripts/import_caffe/).

Caffe -> Frozen (MMdnn)

Microsoft's new MMdnn tool is also a promising way to convert models to TensorFlow. There was an issue (Microsoft/MMdnn#117) where someone was trying to do Caffe -> Frozen like we want to.

• I found this tool difficult to use and couldn't successfully convert models with it.

[colah]

Modelzoo Resources

Eventually, it would be nice to have a list of models with:

• model graphs
• pre-computed neuron visualizations
• links to papers

[colah]

I've been trying to mass export the tensorflow slim models. It appears that some of the provided checkpoints are not compatible with their present code (tensorflow/models#4938).

• ✅ inception_v1
• ✅ inception_v2
• ✅ inception_v3
• ✅ inception_v4
• ✅ inception_resnet_v2
• ❌ resnet_v1_50
• ❌ resnet_v1_101
• ❌ resnet_v1_152
• ✅ resnet_v2_50
• ✅ resnet_v2_101
• ✅ resnet_v2_152
• ❌ vgg_16
• ❌ vgg_19

On the plus side, I have successfully exported 8 more models! :) (Update: now available in #85.)

[colah]

Also, I've got several mostrostities of an ipython-bash fusion script to automatically export models from tf-slim to us. Uh, is that some we'd want to... check in somewhere?? @ludwigschubert ?

• Update: checked into lucid/scratch/scripts in #86.

More generally, the workflows for getting models from different formats into something sane is full of gotchas and workarounds. For now, I'm using this issue as a bit of a log, but it would be good to have institutional memory about how to do this.

[ludwigschubert]

My writeup was/is in this notebook, primarily because I wanted it to be easy for users to put in their own graph and still follow the steps. Let's merge both of our attempts when you have reached a plateau in adding new models.

[colah]

Update: merged #85, adding more models

Despite there likely being some bugs slipping through, I've merged #85. It adds many more models, both from tf-slim and caffe (see list below).

Some potential problems -- at the moment, I'm inferring all of these from the convention of the framework the model was trained in, but this may not be reliable:

• ⚠️ Some models may not have the correct list of labels.
• ⚠️ Presently assuming all caffe models are BGR but it's possible some aren't in cases where they were originally trained in a different framework.
• ⚠️ Some models may not have the correct input range.
• 🏗️ Layer lists for some models are empty.

There are more caffe models that we did not merge.

In addition to the models, the new PR adds scripts at lucid/scratch/scripts/import_slim and lucid/scratch/scripts/import_caffe demonstrating how to import models of those types.

TF Slim Models

• ✅ inception_v1
• ✅ inception_v2
• ✅ inception_v3
• ✅ inception_v4
• ✅ inception_resnet_v2
• ✅ resnet_v1_50
• ✅ resnet_v1_101
• ✅ resnet_v1_152
• ✅ resnet_v2_50
• ✅ resnet_v2_101
• ✅ resnet_v2_152
• ✅ vgg_16
• ✅ vgg_19
• ✅ mobilenet_v1
• ✅ mobilenet_v1_050
• ✅ mobilenet_v1_025
• ✅ nasnet-a_mobile
• ✅ nasnet-a_large
• ✅ pnasnet-5_large
• ✅ pnasnet-5_mobile

Caffe Models

• ✅ AlexNet
• ✅ AlexNet (Places 365)
• ✅ InceptionV1
• ✅ InceptionV1 (Places 205)
• ✅ InceptionV1 (Places 365)
• ✅ CaffeNet
• ✅ VGG_16
• ✅ VGG_19