NVIDIA-Merlin / Transformers4Rec

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation and works with PyTorch.
https://nvidia-merlin.github.io/Transformers4Rec/main
Apache License 2.0
1.07k stars 142 forks source link

Fix recall metric calculation #779

Open sungho-ham opened 2 months ago

sungho-ham commented 2 months ago

Goals :soccer:

Implementation Details :construction:

https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/23d5e3ba73b3d490400e45bf4feba94ed473432f/transformers4rec/torch/ranking_metric.py#L134 The squeeze() function removes all dimensions when batch size is 1. I've added dim=1 argument to keep dimension along batch.

Testing Details :mag:

Test code for error reproduction

import torch
from transformers4rec.torch import ranking_metric

ks = [1, 2]
scores = torch.tensor( [ [1,2,3], ])
labels = torch.tensor( [ [0,1,0], ])
ranking_metric.RecallAt()._metric(ks, scores, labels)

Error message

    135 rel_indices = (num_relevant != 0).nonzero().squeeze()
    136 rel_count = num_relevant[rel_indices]
--> 138 if rel_indices.shape[0] > 0:
    139     for index, k in enumerate(ks):
    140         rel_labels = topk_labels[rel_indices, : int(k)]

IndexError: tuple index out of range
copy-pr-bot[bot] commented 2 months ago

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

rnyak commented 3 days ago

rerun tests