LibCity / Bigscity-LibCity

LibCity: An Open Library for Urban Spatial-temporal Data Mining
https://libcity.ai/
Apache License 2.0
937 stars 168 forks source link

Question about the evaluation metrics in traj_loc_pred task #355

Closed SunderlandAJ-1130 closed 1 year ago

SunderlandAJ-1130 commented 1 year ago

Dear all, thanks for your useful codes.

I have a small question about the evaluation metric about Precision@K in your codes. I am not sure why you only use "self.intermediate_result['total'] * self.topk" to calculate Precision@K and remove "self.topk" in the remaining metrics (e.g., Recall@K).

Thanks!

https://github.com/LibCity/Bigscity-LibCity/blob/f8e5e49a162cb4c5660f655b43d9c0a919a21193/libcity/evaluator/traj_loc_pred_evaluator.py#L62-L71

WenMellors commented 1 year ago

Hello, these metrics are calculated as described in the document https://bigscity-libcity-docs.readthedocs.io/en/latest/user_guide/evaluator/traj_loc_pred.html. You can look up "Recall and Precision at k for Recommender Systems" to understand how these metrics are computed. In essence, you treat the actual next location visited by the user as the item they purchased, and the model's top k predicted locations as the top k recommended items.

SunderlandAJ-1130 commented 1 year ago

Thanks for your generous explanation. We will cite your paper correctly in our current study. Thanks!