[💡SUG] Fair comparison between Recbole and other library

[trannhuthuat96]

Hi team,

When calculating metric on test dataset in autoencoder-based model, e.g., MacridVAE, Recbole sets negative infinity scores for historical items. The list of historical items for one user includes both historical item in training set and validation set.

However, other library only includes the list of items in training set as historical set of items for one user.

This difference causes a big problem when comparing models implementing by different libraries.

Is there any way to only include items in training data as historical items for one user in Recbole?

Thanks,

[Ethan-TZ]

@trannhuthuat96 Hello, thanks for your attention to RecBole! At present, we do not provide an interface to do this, but we will consider adding this option in the subsequent versions. If you want to only include items in training data as historical items for one user, you can change the following code: https://github.com/RUCAIBox/RecBole/blob/7438bef31ea5a27af1aa87d2bd1c215f57e701b9/recbole/data/utils.py#L264 to:

        test_sampler = sampler.set_phase('train')
        test_sampler.phase = 'test'

[trannhuthuat96]

Hi @chenyuwuxin ,

Thanks for your suggestion.