search

yanghu819 / Graph-MLP

73 stars 21 forks source link

Robust experiment #5

Open xuhaiyun43 opened 2 years ago

xuhaiyun43 commented 2 years ago

Hello!I would like to ask what is your specific parameter setting of robustness experiment? I did not reproduce your results with the parameter settings provided in the article. Looking forward to your reply.

yanghu819 commented 2 years ago

I think our paper maybe misleading in writing robust experiment, for the noise is added in testing time(of course the mlp based Graph-mlp will not be influenced for there is no adj input in testing time)

xuhaiyun43 commented 2 years ago

As you said, the Graph-MLP will not be affected because there is no adj input in testing time. But the robustness experimental chart  of the paper is not a straight line, which I think is inaccurate.

------------------ 原始邮件 ------------------ 发件人: "yanghu819/Graph-MLP" @.>; 发送时间: 2021年11月10日(星期三) 下午2:16 @.>; @.**@.>; 主题: Re: [yanghu819/Graph-MLP] Robust experiment (Issue #5)

I think our paper maybe misleading in writing robust experiment, for the noise is added in testing time(of course the mlp based Graph-mlp will not be influenced for there is no adj input in testing time)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

yanghu819 commented 2 years ago

It is not a straight for it runs with no fixed random seeds (In training we don't put on noise).

xuhaiyun43 commented 2 years ago

Thank you very much for your reply. Your paper idea is great!

------------------ 原始邮件 ------------------ 发件人: "yanghu819/Graph-MLP" @.>; 发送时间: 2021年11月10日(星期三) 晚上7:52 @.>; @.**@.>; 主题: Re: [yanghu819/Graph-MLP] Robust experiment (Issue #5)

It is not a straight for it runs with no fixed random seeds (In training we don't put on noise).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

xuhaiyun43 commented 2 years ago

But training without noise and testing with noise always feel strange. In general, we do robustness experiments that both training and testing have the same noise.

------------------ 原始邮件 ------------------ 发件人: "yanghu819/Graph-MLP" @.>; 发送时间: 2021年11月10日(星期三) 晚上7:52 @.>; @.**@.>; 主题: Re: [yanghu819/Graph-MLP] Robust experiment (Issue #5)

It is not a straight for it runs with no fixed random seeds (In training we don't put on noise).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

xuhaiyun43 commented 2 years ago

After careful study of the paper, there is a doubt about the explanation of the paper's Ncontrast loss. I think the denominator should be the complement of the adjacency matrix(A^c).

------------------ 原始邮件 ------------------ 发件人: "yanghu819/Graph-MLP" @.>; 发送时间: 2021年11月10日(星期三) 晚上7:52 @.>; @.**@.>; 主题: Re: [yanghu819/Graph-MLP] Robust experiment (Issue #5)

It is not a straight for it runs with no fixed random seeds (In training we don't put on noise).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Blank-z0 commented 1 year ago

Hello! After reading the paper's section 4.3 Robustness against Corrupted Connection in Inference and this issue, I've reproduced your results. But I have a question, since the Graph-MLP does not need adj as input in testing. Is it necessary to compare the robustness between Graph-MLP and GCN(which need adj as an input)? While the difference between Graph-MLP and GCN is the whether adj is taken as input, it's not fair to compare the robustness under the condition of corrupted adj. BTW, only add noise (corrupt adj) in testing is strange, since training needs adj, it's meaningless to add noise only in testing. Beacese if you don's have an accurate adj (not corrupted), how to get an effective Graph-MLP model in training?