Question: About LightGCN eval mode.

[downeykking]

Hello, thanks for your wonderful work! I have a question about LightGCN eval mode.

To my understanding, in recbole, we have eval methods like full, unixxx like https://recbole.io/docs/user_guide/config/evaluation_settings.html.

If we use full eval, we will call function def full_sort_predict(self, interaction): to get users with all items ratings and get metrics like recall, mrr and so on. If we use unixxx eval, we will call function def predict(self, interaction): and get users with specific items other than all items ratings and get metrics. Do I understand correctly?

But in https://github.com/RUCAIBox/RecBole-GNN/blob/main/recbole_gnn/model/general_recommender/lightgcn.py#L123. If i want to use full eval to evaluate my model, will we always use the previous embedding of the model and get the same metrics in every epoch? Or do I misunderstanding something?

    def full_sort_predict(self, interaction):
        user = interaction[self.USER_ID]
        if self.restore_user_e is None or self.restore_item_e is None:
            self.restore_user_e, self.restore_item_e = self.forward()
        # get user embedding from storage variable
        u_embeddings = self.restore_user_e[user]

        # dot with all item embedding to accelerate
        scores = torch.matmul(u_embeddings, self.restore_item_e.transpose(0, 1))

        return scores.view(-1)

[hyp1231]

Hi, thanks for your attention!

For the first question, yes, the way you described full and unixxx is exactly what we have implemented in RecBole.

For the second question, in the first batch of the evaluation epoch x, we will store the user and item embeddings in restore_user_e and restore_item_e by calling forward, respectively.

https://github.com/RUCAIBox/RecBole-GNN/blob/77d76b9e19436220b32fe61af95ffd95f17c1db7/recbole_gnn/model/general_recommender/lightgcn.py#L123-L126

In the next following batches in evaluation epoch x, we will use the same user and item embeddings, without calling forward again to save time.

Then, in the first batch of training epoch x+1, we will call calculate_loss, and the restore_user_e and restore_item_e will be set as None.

https://github.com/RUCAIBox/RecBole-GNN/blob/77d76b9e19436220b32fe61af95ffd95f17c1db7/recbole_gnn/model/general_recommender/lightgcn.py#L83-L86

Then after training epoch x+1, in the evaluation epoch x+1, we will calculate the value of restore_user_e and restore_item_e again. As a result, we will use the same values of these embeddings within an evaluation epoch but will update these embeddings in the next epoch.

[downeykking]

Hi, thanks for your attention!

For the first question, yes, the way you described full and unixxx is exactly what we have implemented in RecBole.

For the second question, in the first batch of the evaluation epoch x, we will store the user and item embeddings in restore_user_e and restore_item_e by calling forward, respectively.

https://github.com/RUCAIBox/RecBole-GNN/blob/77d76b9e19436220b32fe61af95ffd95f17c1db7/recbole_gnn/model/general_recommender/lightgcn.py#L123-L126

In the next following batches in evaluation epoch x, we will use the same user and item embeddings, without calling forward again to save time.

Then, in the first batch of training epoch x+1, we will call calculate_loss, and the restore_user_e and restore_item_e will be set as None.

https://github.com/RUCAIBox/RecBole-GNN/blob/77d76b9e19436220b32fe61af95ffd95f17c1db7/recbole_gnn/model/general_recommender/lightgcn.py#L83-L86

Then after training epoch x+1, in the evaluation epoch x+1, we will calculate the value of restore_user_e and restore_item_e again. As a result, we will use the same values of these embeddings within an evaluation epoch but will update these embeddings in the next epoch.

I get it! Thanks for your reply. Wish you a good day :)