Inconsistent total image classification time

[psyhtest]

Due to warm up effects, we usually suggest to run our image classification clients with --env.CK_BATCH_COUNT=2. As we explain:

When using the batch count of N, the program classifies N images, but the slow first run is not taken into account when computing the average classification time

This works a treat for the average classification time, but leads to an inconsistency when reporting the total classification time: tflite and tf-cpp truthfully include the time of the first batch into the total, while tf-py and onnx do not:

• tflite

  
  ($ ck benchmark program:image-classification-tflite \
  --repetitions=10 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=2
  ...
  Processing batches...

Batch 1 of 2

Batch loaded in 0.00802251 s Batch classified in 0.16831 s

Batch 2 of 2

Batch loaded in 0.00776105 s Batch classified in 0.0762354 s ... Summary:

Graph loaded in 0.000663s All images loaded in 0.015784s All images classified in 0.244545s Average classification time: 0.076235s Accuracy top 1: 0.5 (1 of 2) Accuracy top 5: 1.0 (2 of 2)

- [`tf-cpp`](https://github.com/dividiti/inference/tree/edge-object_classification-mobilenets/edge/object_classification/mobilenets/tf-cpp)

$ ck benchmark program:image-classification-tf-cpp \ --repetitions=10 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=2 ... Processing batches...

Batch 1 of 2 Batch loaded in 0.00341696 s Batch classified in 0.355268 s

Batch 2 of 2 Batch loaded in 0.00335902 s Batch classified in 0.0108837 s ... Summary:

Graph loaded in 0.053440s All images loaded in 0.006776s All images classified in 0.366151s Average classification time: 0.010884s Accuracy top 1: 0.5 (1 of 2) Accuracy top 5: 1.0 (2 of 2)

- [`tf-py`](https://github.com/dividiti/inference/tree/edge-object_classification-mobilenets/edge/object_classification/mobilenets/tf-py#benchmark-the-performance)

$ ck benchmark program:image-classification-tf-py \ --repetitions=10 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=2 ... Weights loaded in 0.293122s

Batch 1 of 2 Batch loaded in 0.001036s Batch classified in 0.121501s

Batch 2 of 2 Batch loaded in 0.001257s Batch classified in 0.013995s ... Summary:

Graph loaded in 1.115745s All images loaded in 0.002293s All images classified in 0.013995s Average classification time: 0.013995s Accuracy top 1: 0.5 (1 of 2) Accuracy top 5: 1.0 (2 of 2)

- [`onnx`](https://github.com/dividiti/inference/tree/edge-object_classification-mobilenets/edge/object_classification/mobilenets/onnx#benchmark-the-performance)

$ ck benchmark program:image-classification-onnx-py --cmd_key=preprocessed \ --repetitions=10 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=2 ... Batch 1 of 2 Batch loaded in 0.001307s Batch classified in 0.186297s

Batch 2 of 2 Batch loaded in 0.000721s Batch classified in 0.029533s ... Summary:

Graph loaded in 0.018409s All images loaded in 0.002028s All images classified in 0.029533s Average classification time: 0.029533s Accuracy top 1: 0.5 (1 of 2) Accuracy top 5: 1.0 (2 of 2)

[psyhtest]

To avoid any doubt, we should fix onnx-py and tf-py :).

[psyhtest]

Fixed by #24 and https://github.com/ctuning/ck-tensorflow/pull/123.