How to use ssd-mobilenet 300x300 (torch version) provided by mlcommons for inference process

[lixiaolx]

I would like to ask, how to use the ssd-mobilenet model provided by MLcommons on pytorch and convert it into the corresponding jit model, can you provide a demo or link?

[pgmpablo157321]

@lixiaolx Checking the reference implementation, I can see that unfortunately this model is not implemented. Since it was removed in version 2.1, there is no incentive to fix it. There a few steps you can follow to run this benchmark, but you'll have to modify some code.

1. Clone the repository with its submodules:

   git clone --recurse-submodules https://github.com/mlcommons/inference.git --depth 1

2. Install LoadGen

   cd inference/loadgen
   CFLAGS="-std=c++14" python setup.py develop
   cd ..

3. Go to the vision/classification_and_detection folder and download the model and the dataset there. The dataset in its own folder. Download model:

   cd vision/classification_and_detection
   wget https://zenodo.org/record/3239977/files/ssd_mobilenet_v1.pytorch

   Download dataset:

   mkdir data
   cd data
   wget http://images.cocodataset.org/zips/val2017.zip
   wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
   unzip val2017.zip
   unzip annotations_trainval2017.zip

4. Open the file vision/classification_and_detection/run_common.sh and add the following code in the pytorch section:

   if [ $name == "ssd-mobilenet-pytorch" ] ; then
   model_path="$MODEL_DIR/ssd_mobilenet_v1.pytorch"
   profile=ssd-mobilenet-pytorch
   fi

5. With this setup you should be able to run the benchmark from a jupyter notebook. The notebook vision/classification_and_detection/GettingStarted.ipynb contains most of the code for this, but the examples it has are very different. You can run the benchmark from a notebook in the folder vision/classification_and_detection with the following cells:

   import os
   root = os.getcwd()
   os.environ['MODEL_DIR'] = root
   os.environ['DATA_DIR'] = os.path.join(root, "data")
   os.environ['EXTRA_OPS'] ="--time 600"

   Inside of the jupyter notebook, you can run the performance benchmark:

   !./run_local.sh pytorch ssd-mobilenet [device] --scenario [Scenario]

   Inside of the jupyter notebook, you can compute the accuracy:

   !./run_local.sh pytorch ssd-mobilenet [device] --accuracy

   Where device is cpu or gpu