search

yimingxu24 / CLDG

ICDE2023-CLDG: Contrastive Learning on Dynamic Graphs
13 stars 0 forks source link

Running Result Cannot Achieve That on Paper #1

Open QI-Joe opened 6 days ago

QI-Joe commented 6 days ago

Hi, I am currently using your source code on the node classification task as a baseline for testing. However, I noticed that when I ran the code on the provided datasets without any modification, the generated result did not match the scores on the paper. May I ask if there is anything on the code that needs to be changed?

I've attached the result I got; I'm looking forward to your reply.

Paper Name: "CLDG: Contrastive Learning on Dynamic Graphs" Dataset: DBLP, Bitcoinotc and Tax51 Task: Node Classification

Dataset: dblp Hidden dimensions: 128 number of hidden layers: 2 Fanout list: [20, 20] Batch size: 256 GPU: 0 Number of workers per GPU: 4 Max number of epochs: 200 Strategy: sequential (change to low-overlap didn't help much)

Min Contrastive Loss: 9.683, 
Micro-F1: 0.4516,     
Weighted-F1: 0.4375

Dataset: bitcoinotc Hidden dimensions: 128 number of hidden layers: 2 Fanout list: [10, 10] Batch size: 256 GPU: 0 Number of workers per GPU: 4 Max number of epochs: 25 Strategy: sequential 

Min Contrastive Loss: 2.586000
Micro-F1: 0.454200,     
Weighted-F1: 0.416000

Dataset: tax51 Hidden dimensions: 128 number of hidden layers: 2 Fanout list: [20, 20] Batch size: 256 GPU: 0 Number of workers per GPU: 4 Max number of epochs: 200 Strategy: random 

Min Contrastive Loss: 8.986400
Micro-F1: 0.3960   
Weighted-F1: 0.3083
QI-Joe commented 6 days ago

Plus, the code I'm using is in folder CLDG instead of CLDG++

yimingxu24 commented 6 days ago

Thank you for your interest in our work.

Based on the provided information, it seems there might be a discrepancy in your parameter settings. For the DBLP dataset, the epoch should be set to 200, whereas for bitcoinotc, it should be set to 25. Now there may be underfitting and overfitting.

Please confirm if your environment is consistent with the dependencies in our documentation. Different environments and server specifications can indeed result in variations. For clarity and fairness, you can consider specifying your environment and the results you obtained in your paper.

Best regards