search

HKUDS / MMSSL

[WWW'2023] "MMSSL: Multi-Modal Self-Supervised Learning for Recommendation"
https://arxiv.org/abs/2302.10632
155 stars 20 forks source link

About the baselines for MMSSL. #13

Closed RSnewbie closed 9 months ago

RSnewbie commented 9 months ago

Thank you very much for your team's excellent work;

There are some confusion about the baselines of this paper. Is the SGL, LightGCN covered in the paper implemented using https://github.com/HKUDS/SSLRec?

When I ran the tiktok dataset with SGL in SSLRec, the final result was surprisingly good and surpassed most of the baselines. key parameters: {'keep_rate': 0.5, 'layer_num': 3, 'reg_weight': 1e-05, 'cl_weight': 1.0, ' temperature': 0.5 'embedding_size': 32, 'augmentation': 'edge_drop'} Test set [recall@10: 0.0577 recall@20: 0.0856 ] Test set [ndcg@10: 0.0321 ndcg@20: 0.0391 ]

Very much looking forward to your reply, sincerely.

weiwei1206 commented 9 months ago

Thank you very much for your team's excellent work;

There are some confusion about the baselines of this paper. Is the SGL, LightGCN covered in the paper implemented using https://github.com/HKUDS/SSLRec?

When I ran the tiktok dataset with SGL in SSLRec, the final result was surprisingly good and surpassed most of the baselines. key parameters: {'keep_rate': 0.5, 'layer_num': 3, 'reg_weight': 1e-05, 'cl_weight': 1.0, ' temperature': 0.5 'embedding_size': 32, 'augmentation': 'edge_drop'} Test set [recall@10: 0.0577 recall@20: 0.0856 ] Test set [ndcg@10: 0.0321 ndcg@20: 0.0391 ]

Very much looking forward to your reply, sincerely.

Hi! Thank you for your interest in SSLRec! SSLRec does not include the implementation of multi-modal recommendation scenarios. The performance of the implemented SGL in SSLRec is suitable for general collaborative filtering task. It's important to note that there are variations in the details across different tasks, especially those not currently included in SSLRec. These details encompass aspects such as training strategy, data processing, and test protocols. To ensure a fair comparison between different models, it is crucial to use the same settings. Different settings may yield different performance even for the same model. Nevertheless, it's great to hear that you have achieved better performance with SGL using SSLRec. This demonstrates the effectiveness of our SSLRec framework. However, if you aim to reproduce the results in the field of multi-modal recommendation, it would be advisable to consider code frameworks like LATTICE and MMSSL. Only under the same settings can the performance outputs be compared fairly.

RSnewbie commented 9 months ago

Thank you very much for your team's excellent work; There are some confusion about the baselines of this paper. Is the SGL, LightGCN covered in the paper implemented using https://github.com/HKUDS/SSLRec? When I ran the tiktok dataset with SGL in SSLRec, the final result was surprisingly good and surpassed most of the baselines. key parameters: {'keep_rate': 0.5, 'layer_num': 3, 'reg_weight': 1e-05, 'cl_weight': 1.0, ' temperature': 0.5 'embedding_size': 32, 'augmentation': 'edge_drop'} Test set [recall@10: 0.0577 recall@20: 0.0856 ] Test set [ndcg@10: 0.0321 ndcg@20: 0.0391 ] Very much looking forward to your reply, sincerely.

Hi! Thank you for your interest in SSLRec! SSLRec does not include the implementation of multi-modal recommendation scenarios. The performance of the implemented SGL in SSLRec is suitable for general collaborative filtering task. It's important to note that there are variations in the details across different tasks, especially those not currently included in SSLRec. These details encompass aspects such as training strategy, data processing, and test protocols. To ensure a fair comparison between different models, it is crucial to use the same settings. Different settings may yield different performance even for the same model. Nevertheless, it's great to hear that you have achieved better performance with SGL using SSLRec. This demonstrates the effectiveness of our SSLRec framework. However, if you aim to reproduce the results in the field of multi-modal recommendation, it would be advisable to consider code frameworks like LATTICE and MMSSL. Only under the same settings can the performance outputs be compared fairly.

Thank you very much for your reply;

In my current work, four datasets from MMSSL are used as a general collaborative filtering task. In order to compare SGL with my current work more accurately and fairly, can you please send me your code about the SGL implementation in a code framework like MMSSL?

Sincerely.

weiwei1206 commented 8 months ago

Thank you very much for your team's excellent work; There are some confusion about the baselines of this paper. Is the SGL, LightGCN covered in the paper implemented using https://github.com/HKUDS/SSLRec? When I ran the tiktok dataset with SGL in SSLRec, the final result was surprisingly good and surpassed most of the baselines. key parameters: {'keep_rate': 0.5, 'layer_num': 3, 'reg_weight': 1e-05, 'cl_weight': 1.0, ' temperature': 0.5 'embedding_size': 32, 'augmentation': 'edge_drop'} Test set [recall@10: 0.0577 recall@20: 0.0856 ] Test set [ndcg@10: 0.0321 ndcg@20: 0.0391 ] Very much looking forward to your reply, sincerely.

Hi! Thank you for your interest in SSLRec! SSLRec does not include the implementation of multi-modal recommendation scenarios. The performance of the implemented SGL in SSLRec is suitable for general collaborative filtering task. It's important to note that there are variations in the details across different tasks, especially those not currently included in SSLRec. These details encompass aspects such as training strategy, data processing, and test protocols. To ensure a fair comparison between different models, it is crucial to use the same settings. Different settings may yield different performance even for the same model. Nevertheless, it's great to hear that you have achieved better performance with SGL using SSLRec. This demonstrates the effectiveness of our SSLRec framework. However, if you aim to reproduce the results in the field of multi-modal recommendation, it would be advisable to consider code frameworks like LATTICE and MMSSL. Only under the same settings can the performance outputs be compared fairly.

Thank you very much for your reply;

In my current work, four datasets from MMSSL are used as a general collaborative filtering task. In order to compare SGL with my current work more accurately and fairly, can you please send me your code about the SGL implementation in a code framework like MMSSL?

Sincerely.

Could you please provide me with a non-anonymous email? I will send the implementation code to your non-anonymous email.