Experimental Results on Gowalla dataset

[trannhuthuat96]

Hello,

Thank you so much for your very useful work.

I have a question about results on Gowalla dataset, based on the log in https://github.com/openbenchmark/BARS/tree/master/candidate_matching/benchmarks/SimpleX/SimpleX_gowalla_x1

How do you divide training data into training and validation sets? From the log, test set is currently treated as validation set and reported results are selected as the best iteration on test set, which causes test data leakage. A similar problem was reported in this paper https://arxiv.org/pdf/2005.09683.pdf?

Look forward to hearing from you soon, Thank you so much.

[zhujiem]

Yes, we follow the original data split with only train/test subsets and thus the results can be seen as validation accuracy.

[trannhuthuat96]

Got it, thanks!