ctuning / ck-mlperf

This repository is outdated! Join the open MLPerf workgroup to participate in the development of the next generation of automation workflows for MLPerf benchmarks:
https://bit.ly/mlperf-edu-wg
BSD 3-Clause "New" or "Revised" License
32 stars 23 forks source link

Implement MLPerf ResNet50 package for TensorFlow #2

Closed psyhtest closed 5 years ago

psyhtest commented 5 years ago

The MLPerf ResNet50 reference by @guschmue supports weights both in the TensorFlow and ONNX formats.

@guschmue notes:

The benchmark is a reference implementation that is not meant to be the fastest implementation possible. It is written in python which might make it less suitable for lite models like mobilenet or large number of cpu's. We are thinking to provide a c++ implementation with identical functionality in the near future.

In fact, we already have such an implementation wrapped in CK: ck-tensorflow:program:image-classification-tf-cpp. Let's check it works with the ResNet50 weights in the TensorFlow format by providing a CK package.

psyhtest commented 5 years ago

I vaguely recall that the CK implementation uses *.ckpt files, while the reference one needs a *.pb file. However, the official TF ResNet50 page also mentions *.ckpt. It appears we need the checkpoint listed as "ResNet-50 v1 (fp32, Accuracy 76.53%)" ("v1" means "v1.5", as explained there).

psyhtest commented 5 years ago

We should also check if SavedModels is something we might want to support as well (perhaps to be converted to TFLite or to proprietary inference engines).

psyhtest commented 5 years ago

For reference, this is how program:image-classification-tf-cpp is used in the MLPerf MobileNet reference.

ens-lg4 commented 5 years ago

The package ck-tensorflow:package:tensorflowmodel-resnet50-v1.5-mlperf has been implemented and program:image-classification-tf-cpp agrees to take it as input.