Hi there!
It might be useful to group some metrics together to have less code and more coherence. My use case was using some ignite metrics in HuggingFace Trainer API which expects a single callable for computing metrics. Having this feature, that could be easily realizable with Ignite. MetricGroup could be something like this:
from typing import Any, Dict
from ignite.metrics import Metric
class MetricGroup(Metric):
_state_dict_all_req_keys = ('metrics',)
def __init__(self, metrics:Dict[str, Metric]):
self.metrics = metrics
super(MetricGroup, self).__init__()
def reset(self):
for m in self.metrics.values():
m.reset()
def update(self, output):
for m in self.metrics.values():
m.update(m._output_transform(output))
def compute(self) -> Dict[str, Any]:
return {k: m.compute() for k,m in self.metrics.items()}
Usage:
metric = MetricGroup({'acc': Accuracy(), 'perplexity': Perplexity()})
metric.attach(engine) # or sth in HF Trainer API
🚀 Feature
Hi there! It might be useful to group some metrics together to have less code and more coherence. My use case was using some ignite metrics in HuggingFace Trainer API which expects a single callable for computing metrics. Having this feature, that could be easily realizable with Ignite.
MetricGroup
could be something like this:Usage: