DSE-MSU / DeepRobust

A pytorch adversarial library for attack and defense methods on images and graphs
MIT License
957 stars 191 forks source link

About Random Seeds #147

Open L1J1aJun opened 10 months ago

L1J1aJun commented 10 months ago

Why do I get different results when I set random seeds before and after the dataset? Taking the GCN model as an example, using the CORA dataset after meta attack (ptd_rate=0.05), the model's performance differs greatly when I set random seeds before and after the dataset.

ChandlerBang commented 10 months ago

I am not exactly sure about the described scenario, but we are supposed to use the same data split before and after attack. This ensures that the attacker uses the correct supervision to learn the perturbation. Otherwise, the attack is very likely to degrade the model performance.

L1J1aJun commented 10 months ago

我不完全确定所描述的场景,但我们应该在攻击前后使用相同的数据拆分。这可确保攻击者使用正确的监督来学习扰动。否则,攻击很可能会降低模型性能。

感谢你的回复,我发现这个错误发生在使用nettack划分的时候,但是prognn划分不会出现这个问题