How to reproduce the result of Head, Mid, Tail

[LeavesLi1015]

I'm working on reproduce the performance of Head, Mid, Tail in your work, and I came across a problem. I don't no how to correctly create a ProxyEvaluator using group_view and eval_test_ood_split.

I create a ProxyEvaluator by the following:

eval_test_id_grouped_0 = ProxyEvaluator(data, data.train_user_list, eval_test_id_split[0], group_view=grp_view, top_k=[20])
eval_test_id_grouped_1 = ProxyEvaluator(data, data.train_user_list, eval_test_id_split[1], group_view=grp_view, top_k=[20])
eval_test_id_grouped_2 = ProxyEvaluator(data, data.train_user_list, eval_test_id_split[2], group_view=grp_view, top_k=[20])

Error merges like this, Could you help me out?

valid:{'recall': 0.022834795, 'hit_ratio': 0.03245308, 'precision': 0.0016720997, 'ndcg': 0.0107858125, 'mrr': 0.009067356, 'map': 0.008989009}
test_id:{'recall': 0.022585321, 'hit_ratio': 0.03614483, 'precision': 0.0018866506, 'ndcg': 0.011409973, 'mrr': 0.01073817, 'map': 0.010626127}
test_ood:{'recall': 0.0011533491, 'hit_ratio': 0.002379852, 'precision': 0.00012707955, 'ndcg': 0.0005133333, 'mrr': 0.00051824056, 'map': 0.00050651364}
Traceback (most recent call last):
  File "main.py", line 792, in <module>
    is_best, temp_flag = evaluation(args, data, model, epoch, base_path, evaluator, eval_names[i])
  File "main.py", line 155, in evaluation
    ret, _ = evaluator.evaluate(model)
  File "/home/user/lhx/GNNcoding/BC_loss/evaluator/proxy_evaluator.py", line 109, in evaluate
    return self.evaluator.evaluate(model)
  File "/home/user/lhx/GNNcoding/BC_loss/evaluator/grouped_evaluator.py", line 109, in evaluate
    tmp_result = self.evaluator.evaluate(model, users)
  File "/home/user/lhx/GNNcoding/BC_loss/evaluator/backend/cpp/uni_evaluator.py", line 146, in evaluate
    test_items = [self.user_pos_test[u] for u in batch_users]
  File "/home/user/lhx/GNNcoding/BC_loss/evaluator/backend/cpp/uni_evaluator.py", line 146, in <listcomp>
    test_items = [self.user_pos_test[u] for u in batch_users]
TypeError: 'NoneType' object is not subscriptable

[Arthurma71]

The third argument for ProxyEvaluator accepts a dict with test user as key and a list containing test items for that user as value, just like the format of data.test_id_user_list. Check if each eval_test_id_split[i] contains test item in that format.