Currently each setting ("dataset") comes with its own Metric. This uses the PyTorch Lightning Metric API (https://torchmetrics.readthedocs.io/en/stable/). It might be, that this API is outdated, and it might also be that Metrics are consistent across datasets, hence leading to duplicity.
Goal:
A better API for metrics, that is added to the Task part, like the losses.
Currently each setting ("dataset") comes with its own Metric. This uses the PyTorch Lightning Metric API (https://torchmetrics.readthedocs.io/en/stable/). It might be, that this API is outdated, and it might also be that Metrics are consistent across datasets, hence leading to duplicity.
Goal:
Culprits: