Questions about hyperparameters.

[Leirunlin]

Hi! Glad to see the great work and code. I have some questions about the setting and hyperparams.

• What do the 'prune_graph' and 'mul_run' params mean, and when should they be used?
• Are the results conducted on the same split with multiple repeats or multiple splits? I see both '--runs' and '--mul-run' are included.
• It seems that sepitml is not enhanced with HAO in run_cora.sh. Why?

I also found two bugs in the code:

• The following line returns an error with the latest version of Scipy. "AttributeError: row not found" https://github.com/LFhase/GIA-HAO/blob/296c5b9025602348091b1efaf9f79bfd58e873b3/attacks/pgd.py#L155 I solved the problem by replacing all csr-matrix with coo format.
• The type of edge weight in adaptive injection should be changed to FloatTensor. https://github.com/LFhase/GIA-HAO/blob/296c5b9025602348091b1efaf9f79bfd58e873b3/attacks/injection.py#L512
• GIA attack is coupled with HAO but not named started with "r". https://github.com/LFhase/GIA-HAO/blob/296c5b9025602348091b1efaf9f79bfd58e873b3/run_cora.sh#L17C5-L17C5.

[LFhase]

Hi @Leirunlin thank you for your interests in our work.

• prune_graph is used to reduce the size of graphs to reduce the memory cost. It's an available function that hasn't been used in our scripts;
• mul_run refers to assigning the specific random seed in the name of the perturbed graph, therefore it enables a robustness evaluation per distinct attack, though we didn't use it following the previous practice (therefore the results should be with the same splits);
• speitml is not supported in our code (we didn't include it because it has little difference from other injection methods) and I'll fix it soon;

Thanks for pointing out the bugs,

• may I know which torch_geometric version you were using?
• gia is actually rpgd, and I will fix it soon.

Please feel free to let me know if you have any further questions : )

[Leirunlin]

Thanks for your reply! @LFhase

• Is prune graph adopted in larger graphs like arxiv?
• In fact, I run speitml smoothly. The injected pattern is multi-layer, and the attack performance is fine. Could you share more details about the problem it has?

I am currently trying to conduct some experiments based on the project, so any attempts or experiences would be valuable.

For the bugs,

• I use the latest torch_geometric, which is (2.4.0). But I think the relevant libraries would be torch (1.12.1) and Scipy (1.8.1). After I used coo format for all sparse matrices, the problem was solved.

[LFhase]

Thank you for your follow-up! @Leirunlin

• We didn't use it for arxiv as it does not cost much memory. Nevertheless, if you wanna run untargeted attack on larger graphs such reddit, you may need it (note that for larger graphs, we recommend targeted attack as it's more reasonable).
• Yes, if you look into the code, the SPEIT class does not include codes for HAO regularization. You may add it if you wanna try it.
• As for the bugs, maybe there's a compatibility issue. HAO was developed with

  grb==0.1.0
  torch==1.9.0
  torch-geometric==1.7.2
  scipy==1.6.2

  Nevertheless, thank you for your fix and I will mention it in the README.

If you have further questions or anything to discuss, feel free to continue the issue, or drop me an email (or email me to exchange other instant communication channels) : )

[1234238]

Thanks for your reply! @LFhase

• Is prune graph adopted in larger graphs like arxiv?
• In fact, I run speitml smoothly. The injected pattern is multi-layer, and the attack performance is fine. Could you share more details about the problem it has?

I am currently trying to conduct some experiments based on the project, so any attempts or experiences would be valuable.

For the bugs,

• I use the latest torch_geometric, which is (2.4.0). But I think the relevant libraries would be torch (1.12.1) and Scipy (1.8.1). After I used coo format for all sparse matrices, the problem was solved.

Hello, after using adj = coo_matrix(adj) in the inject part, the bug is no longer displayed. However, I found a problem: new_edges_x.extend([x, y]) and new_edges_y.extend([y, x]) with new_data.extend([1, 1]) will insert two edges, but if I want to insert a single edge from x to y, using new_edges_x.extend([x]), new_edges_y.extend([y]), and new_data.extend([1]), I find that it seems to still insert two edges. Specifically, adj.coo[:1][0].shape[0] remains unchanged, which confuses me. Do you know why this is happening?

[LFhase]

Thanks for your reply! @LFhase

• Is prune graph adopted in larger graphs like arxiv?
• In fact, I run speitml smoothly. The injected pattern is multi-layer, and the attack performance is fine. Could you share more details about the problem it has?

I am currently trying to conduct some experiments based on the project, so any attempts or experiences would be valuable. For the bugs,

• I use the latest torch_geometric, which is (2.4.0). But I think the relevant libraries would be torch (1.12.1) and Scipy (1.8.1). After I used coo format for all sparse matrices, the problem was solved.

Hello, after using adj = coo_matrix(adj) in the inject part, the bug is no longer displayed. However, I found a problem: new_edges_x.extend([x, y]) and new_edges_y.extend([y, x]) with new_data.extend([1, 1]) will insert two edges, but if I want to insert a single edge from x to y, using new_edges_x.extend([x]), new_edges_y.extend([y]), and new_data.extend([1]), I find that it seems to still insert two edges. Specifically, adj.coo[:1][0].shape[0] remains unchanged, which confuses me. Do you know why this is happening?

Hi @1234238 Could you inspect the matrix of adj line by line, and see if it's been changed somewhere else, e.g., some functions may change the graph from directed to undirected?

[1234238]

Thanks for your reply! @LFhase

• Is prune graph adopted in larger graphs like arxiv?
• In fact, I run speitml smoothly. The injected pattern is multi-layer, and the attack performance is fine. Could you share more details about the problem it has?

I am currently trying to conduct some experiments based on the project, so any attempts or experiences would be valuable. For the bugs,

• I use the latest torch_geometric, which is (2.4.0). But I think the relevant libraries would be torch (1.12.1) and Scipy (1.8.1). After I used coo format for all sparse matrices, the problem was solved.

Hello, after using adj = coo_matrix(adj) in the inject part, the bug is no longer displayed. However, I found a problem: new_edges_x.extend([x, y]) and new_edges_y.extend([y, x]) with new_data.extend([1, 1]) will insert two edges, but if I want to insert a single edge from x to y, using new_edges_x.extend([x]), new_edges_y.extend([y]), and new_data.extend([1]), I find that it seems to still insert two edges. Specifically, adj.coo[:1][0].shape[0] remains unchanged, which confuses me. Do you know why this is happening?

Hi @1234238 Could you inspect the matrix of adj line by line, and see if it's been changed somewhere else, e.g., some functions may change the graph from directed to undirected?

Thank you for your reply. Additionally, I seem to have found another problem. When using meta_attack, for example, using python -u gnn_misg.py --dataset 'cora' --inductive --eval_robo --eval_attack 'seqgia' --n_inject_max 300 --n_edge_max 7 --runs 1 --disguise_coe 1 --use_ln 0 --injection 'meta' --gpu 0 --model 'gat' --sequential_step 1.0 --grb_split, my execution produces the following error:

Traceback (most recent call last): File "gnn_misg.py", line 716, in main() File "gnn_misg.py", line 634, in main x_attack, adj_attack, target_idx = eval_robustness(model, x_test, adj_test, target_idx, data.y, device, args, run) File "gnn_misg.py", line 232, in eval_robustness adj_attack, features_attack = attacker.attack(model=model, File "/home/lc/GIA-HAO/attacks/seqgia.py", line 125, in attack adj_attack = meta_injection(self, model, adj_attack, n_inject_cur, self.n_edge_max, features_tmp, File "/home/lc/GIA-HAO/attacks/injection.py", line 641, in meta_injection adj_meta_grad = torch.autograd.grad(pred_loss, vals, retain_graph=True)[0] File "/home/lc/anaconda3/envs/dgl/lib/python3.8/site-packages/torch/autograd/init.py", line 300, in grad return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior. But what's strange is that GCN can run, but other models cannot run. I don't know if you have encountered this problem?