search

pytorch / torcheval

A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to facilitate metric computation in distributed training and tools for PyTorch model evaluations.
https://pytorch.org/torcheval
Other
214 stars 46 forks source link

Potentially Misleading Error Message for multiclass_precision #198

Open adrianjoshua-strutt opened 2 months ago

adrianjoshua-strutt commented 2 months ago

📚 The doc issue

Hi!

I encountered the following warning message while working with the multiclass_precision function:

WARNING:root:tensor([[1],
        [2],
        [3]]) classes have zero instances in both the predictions and the ground truth labels. Precision is still logged as zero.

Here is a minimal example that produces this warning:

import torch
import torcheval.metrics

print(torcheval.metrics.functional.classification.precision.multiclass_precision(
    input = torch.tensor([
        [1, 0, 0, 0],
        [1, 0, 0, 0],
        [1, 0, 0, 0],
        [1, 0, 0, 0],
    ]),
    target = torch.tensor([
        0, 1, 2, 3
    ]),
    num_classes = 4,
    average = None,
))

I find the message somewhat misleading, as it implies that there are zero instances for certain classes in both predictions and ground truth labels. However, the ground truth labels do contain instances for each class, while only the predictions have zero instances. This led to me thinking that my ground truth labels were incorrect.

Thanks !

Suggest a potential alternative/fix

If I am not getting something wrong here, the message could be changed to:


WARNING:root:tensor([[1],
        [2],
        [3]]) classes have zero instances in either the predictions or the ground truth labels. Precision is still logged as zero.