Question about prediction over test set

Hello!

In issue https://github.com/RUCAIBox/RecBole/issues/1848 I asked about the process of extraction of recommendations for a particular list of user ids and that works perfectly fine.

Now I have a questions about the particular items of a recommendation for a particular user id. My general misundestanding for now relates to the set of items that are actually used in recoms in functions below. Correct me if I do a mistake, but as far as I understand by now the main functions that we use for recommendations for a particular list of users - we do the following:

suppose that we work with original mode, not sequential, then the first step is to construct interaction (user ids) and we achieve this via command (if we do not use sequential recoms)

input_interaction = dataset.join(Interaction({uid_field: uid_series}))

as a results in nu case input_interaction.numpy() gives {'user_id': array([23033, 10512, 609])} so these are id provided by model but not the initial token ids and after turning it to device it is used the following command:

scores = model.full_sort_predict(input_interaction)

that in fact does vector multiplication between particular user embedding and all item embeddings. And since all models use this function to predict rating the approach is equivalent

(for my example I received something like this

tensor([[   -inf,  1.5536, -1.2930,  ..., -2.8303, -3.3913, -2.9431],
        [   -inf,  0.9521, -1.5225,  ..., -2.6522, -2.4662, -3.0087],
        [   -inf,  0.8473, -0.0743,  ..., -1.3093, -2.3306, -0.6975]])
torch.Size([3, 155448])

)

def full_sort_predict(self, interaction):
    user = interaction[self.USER_ID]
    user_e = self.get_user_embedding(user)
    all_item_e = self.item_embedding.weight
    score = torch.matmul(user_e, all_item_e.transpose(0, 1))
    return score.view(-1)

So the question - please could you a little bit clarify the process. I thought that the ideal case is when We have some list of users with their ids, and for them predicts ratings for the items that they did not interact before (since for recommendation purpose we are not interested in recommending an item with which user interacted before). But following this code I have a misunderstanding which set of items we actually use. On the one hand I see that we use dataset = test_data.dataset, that is why it is likely that items from tests are also used, but in fact there are could be items in test that users have interacted already. On the other hand I see all_item_e = self.item_embedding.weight command gives a point of using all the items. Could you please clarify which items are actually using for recommendations and if all items are used what d you think is the efficient way for a particular user to drop items that he or she has interacted before

(For reference)

# @Time   : 2020/12/25
# @Author : Yushuo Chen
# @Email  : chenyushuo@ruc.edu.cn

# UPDATE
# @Time   : 2020/12/25
# @Author : Yushuo Chen
# @email  : chenyushuo@ruc.edu.cn

"""
recbole.utils.case_study
#####################################
"""

import numpy as np
import torch

from recbole.data.interaction import Interaction

@torch.no_grad()
def full_sort_scores(uid_series, model, test_data, device=None):
    """Calculate the scores of all items for each user in uid_series.

    Note:
        The score of [pad] and history items will be set into -inf.

    Args:
        uid_series (numpy.ndarray or list): User id series.
        model (AbstractRecommender): Model to predict.
        test_data (FullSortEvalDataLoader): The test_data of model.
        device (torch.device, optional): The device which model will run on. Defaults to ``None``.
            Note: ``device=None`` is equivalent to ``device=torch.device('cpu')``.

    Returns:
        torch.Tensor: the scores of all items for each user in uid_series.
    """
    device = device or torch.device("cpu")
    uid_series = torch.tensor(uid_series)
    uid_field = test_data.dataset.uid_field
    dataset = test_data.dataset
    model.eval()

    if not test_data.is_sequential:
        input_interaction = dataset.join(Interaction({uid_field: uid_series}))
        history_item = test_data.uid2history_item[list(uid_series)]
        history_row = torch.cat(
            [torch.full_like(hist_iid, i) for i, hist_iid in enumerate(history_item)]
        )
        history_col = torch.cat(list(history_item))
        history_index = history_row, history_col
    else:
        _, index = (dataset.inter_feat[uid_field] == uid_series[:, None]).nonzero(
            as_tuple=True
        )
        input_interaction = dataset[index]
        history_index = None

    # Get scores of all items
    input_interaction = input_interaction.to(device)
    try:
        scores = model.full_sort_predict(input_interaction)
    except NotImplementedError:
        input_interaction = input_interaction.repeat_interleave(dataset.item_num)
        input_interaction.update(
            test_data.dataset.get_item_feature().to(device).repeat(len(uid_series))
        )
        scores = model.predict(input_interaction)

    scores = scores.view(-1, dataset.item_num)
    scores[:, 0] = -np.inf  # set scores of [pad] to -inf
    if history_index is not None:
        scores[history_index] = -np.inf  # set scores of history items to -inf

    return scores

def full_sort_topk(uid_series, model, test_data, k, device=None):
    """Calculate the top-k items' scores and ids for each user in uid_series.

    Note:
        The score of [pad] and history items will be set into -inf.

    Args:
        uid_series (numpy.ndarray): User id series.
        model (AbstractRecommender): Model to predict.
        test_data (FullSortEvalDataLoader): The test_data of model.
        k (int): The top-k items.
        device (torch.device, optional): The device which model will run on. Defaults to ``None``.
            Note: ``device=None`` is equivalent to ``device=torch.device('cpu')``.

    Returns:
        tuple:
            - topk_scores (torch.Tensor): The scores of topk items.
            - topk_index (torch.Tensor): The index of topk items, which is also the internal ids of items.
    """
    scores = full_sort_scores(uid_series, model, test_data, device)
    return torch.topk(scores, k)

RUCAIBox / RecBole

Question about prediction over test set #1857