Metric values are returned out of order

[mrzv]

🐛 Bug

metric.values.values_list() returns values not in the order they were logged. Same true for values_numpy(), sparse_list(), sparse_numpy(). It's not clear how to sort them in the correct order.

To reproduce

losses = [m for m in run.metrics() if m.name == "loss"][0].values.values_list()

Expected behavior

losses appear in the order they were added.

Environment

• Aim Version 3.17.1
• Python version 3.11.2
• pip version 23.0.1
• macOS

[mrzv]

I found a way to fix this, but it seems counter-intuitive. For anyone looking for the same:

indices = np.argsort(metric.timestamps.values_numpy())
losses = losses[indices]

[gorarakelyan]

@mrzv thanks for reporting! This is super weird, will look into it.

[YodaEmbedding]

I found a way to fix this, but it seems counter-intuitive. [...]

I ended up using a similar workaround for out-of-order metric data when I encountered this a few months ago. It works by doing a reverse lookup on a known monotonically increasing metric such as timestamps (as you indicated). In my case, I log an "epoch" metric, which is just the value of the current epoch whenever I log my epoch metrics, and is strictly monotonically increasing, e.g. 1, 2, 3, 4, .... Here's a minimal workaround example:

# Workaround via reverse lookup.
def get_ordered_metric_values(metric, context, monotonic_metric="epoch"):
    _, unordered_metric_values = run.get_metric(metric, context).values.sparse_numpy()
    _, monotonic_metric_values = run.get_metric(monotonic_metric, context).values.sparse_numpy()
    idx = np.argsort(monotonic_metric_values)
    return unordered_metric_values[idx]