Performing Inference in RecBole

[adhiiisetiawan]

I am currently using RecBole for my recommendation task and have successfully trained a model using custom data from atomic files and the NCF/NeuMF algorithm. However, I am struggling to find documentation or resources on how to perform inference with my trained model. Is there any available resource on this matter? Thank you in advance.

[UPDATE]

I try use case_study_example.py and modified with my model. However, when I execute full_sort_topk on line 27, I encounter an error:

AttributeError: 'NegSampleEvalDataLoader' object has no attribute 'uid2history_item'.

I think the error because i use uni100 evaluation mode instead of full. It's proven when i short train a model again with full evaluation mode and ranking based metrics, the inference process run smoothly. But i my case, i wanna use value based metrics, which is can't use full evaluation. Is it possible or any other approach to use full evaluation mode in value based metrics? Because to predict need use full evaluation mode. Or any other approach to keep using uni100 evaluation mode in value based metrics?

The goal is predict/inference using my model.

Here's the error for details

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/home/adhi/emos-recsys/RecBole/test_recbole.ipynb Cell 6 line 1
----> [1](vscode-notebook-cell://wsl%2Bubuntu/home/adhi/emos-recsys/RecBole/test_recbole.ipynb#W6sdnNjb2RlLXJlbW90ZQ%3D%3D?line=0) topk_score, topk_iid_list = full_sort_topk(
      [2](vscode-notebook-cell://wsl%2Bubuntu/home/adhi/emos-recsys/RecBole/test_recbole.ipynb#W6sdnNjb2RlLXJlbW90ZQ%3D%3D?line=1)     uid_series, model, test_data, k=10, device=config["device"]
      [3](vscode-notebook-cell://wsl%2Bubuntu/home/adhi/emos-recsys/RecBole/test_recbole.ipynb#W6sdnNjb2RlLXJlbW90ZQ%3D%3D?line=2) )

File [~/emos-recsys/RecBole/recbole/utils/case_study.py:98](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/adhi/emos-recsys/RecBole/~/emos-recsys/RecBole/recbole/utils/case_study.py:98), in full_sort_topk(uid_series, model, test_data, k, device)
     79 """Calculate the top-k items' scores and ids for each user in uid_series.
     80 
     81 Note:
   (...)
     95         - topk_index (torch.Tensor): The index of topk items, which is also the internal ids of items.
     96 """
     97 scores = full_sort_scores(uid_series, model, test_data, device)
---> 98 return torch.topk(scores, k)

File [~/emos-recsys/.env-recsys/lib/python3.10/site-packages/torch/utils/_contextlib.py:115](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/adhi/emos-recsys/RecBole/~/emos-recsys/.env-recsys/lib/python3.10/site-packages/torch/utils/_contextlib.py:115), in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File [~/emos-recsys/RecBole/recbole/utils/case_study.py:47](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/adhi/emos-recsys/RecBole/~/emos-recsys/RecBole/recbole/utils/case_study.py:47), in full_sort_scores(uid_series, model, test_data, device)
     45 input_interaction = dataset.join(Interaction({uid_field: uid_series}))
     46 history_item = test_data.uid2history_item[list(uid_series)]
---> 47 history_row = torch.cat(
     48     [torch.full_like(hist_iid, i) for i, hist_iid in enumerate(history_item)]
     49 )
     50 history_col = torch.cat(list(history_item))
     51 history_index = history_row, history_col

File [~/emos-recsys/RecBole/recbole/data/dataloader/abstract_dataloader.py:113](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/adhi/emos-recsys/RecBole/~/emos-recsys/RecBole/recbole/data/dataloader/abstract_dataloader.py:113), in AbstractDataLoader.__getattribute__(self, _AbstractDataLoader__name)
    111 if not start_iter and __name == "dataset":
    112     __name = "_dataset"
--> 113 return super().__getattribute__(__name)

AttributeError: 'NegSampleEvalDataLoader' object has no attribute 'uid2history_item'

And here's my config for NeuMF

# Environment Settings
gpu_id: '0'
worker: 16
seed: 2020
checkpoint_dir: 'saved/'
log_wandb: True
wandb_project: 'neural-collaborative-filtering'

# dataset config : General Recommendation
USER_ID_FIELD: user_id
ITEM_ID_FIELD: item_id
load_col:
    inter: [user_id, item_id]

# model config
mf_embedding_size: 32
mlp_embedding_size: 32
mlp_hidden_size: [256, 256, 128, 64]
dropout_prob: 0.1
mf_train: True
mlp_train: True
use_pretrain: False
mf_pretrain_path: None
mlp_pretrain_path: None

# Training and evaluation config
epochs: 600
learner: adam
learning_rate: 0.0001
train_batch_size: 4096
eval_batch_size: 4096
stopping_step: 10
train_neg_sample_args:
    distribution: uniform
    sample_num: 1
    alpha: 1.0
    dynamic: False
    candidate_num: 0
eval_args:
    group_by: user
    order: RO
    split: {'RS': [0.8,0.1,0.1]}
    mode: uni100
metrics: ['AUC', 'MAE', 'RMSE', 'LogLoss']
topk: 10
valid_metric: RMSE
metric_decimal_place: 4

[zhengbw0324]

@adhiiisetiawan Hello, regarding testing the trained model, I suggest you refer to the run_recbole function. After using torch.load() to load the model parameters, complete the test based on trainer.evaluate. In addition, the NeuMF model as a general recommender model is generally only suitable for ranking tasks and ranking evaluation metrics such as NDCG and MRR.

[adhiiisetiawan]

Hello @zhengbw0324 , thank you for your response and explanation. Okay, run_recbole function just for testing right? But, when i want to use for inference, like got list of recommended item for each user, can i use case_study_example.py for reference? I see there about get to topk item for recommendation

[lukas-wegmeth]

@adhiiisetiawan I was also running into this problem. However, assigning a "mode" for the validation and test set separately in the config is possible. Hence, if I set the "mode" for the test set to "full", the case study will work. And I can still use "uni100" for validation. Suppose the model was already trained with the "mode" setting at "uni100". In that case, it is possible to call "create_dataset()" after loading the model with an updated config to transform the test data into the "FullSortEvalDataLoader" object that has the required attribute. However, I dislike this solution because it could have unforeseen consequences. Ideally, you would set the "mode" correctly right away.

[adhiiisetiawan]

hi @lukas-wegmeth thank you for you answer,