comet-ml / issue-tracking

Questions, Help, and Issues for Comet ML
https://www.comet.ml
85 stars 7 forks source link

`API.get_metrics_summary(metric=name)` returns an empty list instead of dict #362

Closed neighthan closed 1 year ago

neighthan commented 4 years ago

If you call API.get_metrics_summary with the name of a particular metric, and the experiment has data for that metric, a dictionary is returned. If the experiment lacks data for that metric, then an empty list is returned. An empty dictionary would make more sense here so that the return type in this case doesn't depend on whether the metric exists or not. This makes further processing easier because you don't have to handle two different possible return types. As an example, here's what I would like to do:

from comet_ml.api import API, Parameter
api = API()
workspace = "..."
project = "..."
query = Parameter("something") == "another thing"

exps = api.query(workspace, project, query)
metric_name = "return"
default = float("nan")
max_metrics = [exp.get_metrics_summary(metric_name).get("valueMax", default) for exp in exps]

but if the metric might not exist on some experiments, then I have to handle empty lists instead by doing something like

max_metrics = []
for exp in exps:
    metric = exp.get_metrics_summary(metric_name)
    if isinstance(metric, list):
        max_metrics.append(default)
    else:
        max_metrics.append(metric["valueMax"])
dsblank commented 4 years ago

Yes, good idea!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 1 year ago

This issue was closed because it has been stalled for 5 days with no activity.