search

usaito / unbiased-implicit-rec-real

(WSDM2020) "Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback"
Apache License 2.0
30 stars 5 forks source link

We can never match the reported values in your two papers. #3

Open AdversarialSelfTraining opened 4 years ago

AdversarialSelfTraining commented 4 years ago

Dear authors

Thank your for sharing the source!

We encountered some issues with reproducing your results and I was wondering if you could assist us. Please find below a few questions:

  1. The results of RelMF on Yahoo dataset reported in your two papers: Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback (WSDM 2020)and Unbiased Pairwise Learning from Biased Implicit Feedback (ICTIR 2020) are different. In fact, there is one order of magnitude difference between them. See:

DCG@3 DCG@5 Recall@3 Recall@5 MAP@3 MAP@5 0.08576 0.09861 0.09249 0.11993 0.07065 0.07887 (ICTIR 2020)

DCG@3 DCG@5 Recall@3 Recall@5 MAP@3 MAP@5 0.3773 0.4590 0.3946 0.5891 0.3269 0.4076 (WSDM 2020)

Could you please let us know why the results of your two papers are different

We look forward to your reply.

Thank you