Mismatch in the order of the column names in the benchmarking script for transformer models

[sayakpaul]

Here the order of the column name is as follows:

https://github.com/microsoft/onnxruntime/blob/007ef427493052ca4a3e90b4560baa8c2985b98d/onnxruntime/python/tools/transformers/benchmark_helper.py#L204-#L226

        column_names = [
            "engine",
            "version",
            "providers",
            "device",
            "precision",
            "optimizer",
            "io_binding",
            "model_name",
            "inputs",
            "threads",
            "batch_size",
            "sequence_length",
            "custom_layer_num",
            "datetime",
            "test_times",
            "QPS",
            "average_latency_ms",
            "latency_variance",
            "latency_90_percentile",
            "latency_95_percentile",
            "latency_99_percentile",
        ]

But the latency results aren't exactly returned in that order:

https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/benchmark_helper.py#L192-#L198

I mean, QPS is at last in the results returned by get_latency_result().

[tianleiwu]

@sayakpaul, order of the columns is by design. You can open the detail.csv file with Excel to view a table like the following:

get_latency_result() returns a dictionary (not OrderedDict), and the order of keys does not matter.

[sayakpaul]

Honestly, I am still not sure. How does any order matter for the CSV file then?