search

zhiming-xu / conad

Contrastive Attributed Network Anomaly Detection with Data Augmentation (PAKDD'22)
24 stars 5 forks source link

Couldn’t handle large data #3

Open Shakib-IO opened 1 year ago

Shakib-IO commented 1 year ago

Hi @zhiming-xu I have been attempting to apply this method on a large dataset containing (1271939) nodes. Could you please provide guidance on how to effectively implement this method for such a dataset? Thank you.

zhiming-xu commented 1 year ago

Hi @Shakib-IO ,

Thank you for your interest in this project. I think it's a common challenge for GNN models to scale up to large datasets. For your cases of a graph with over 1 million nodes, you might consider running it on CPU where more memory is available. For efficiency, GraphSAGE can be used in favor of GAT. Besides, splitting the entire graph into smaller partitions might also work. In general, you might want to search for works in the domain of large graph models and frameworks.