search

tkipf / gcn

Implementation of Graph Convolutional Networks in TensorFlow
MIT License
7.16k stars 2k forks source link

How to distribute gcn #15

Open chunyang-wen opened 7 years ago

chunyang-wen commented 7 years ago

Hi. First of all, Good work bro. I have read your paper and your blog about gcn. One thing that comes to my mind is that: If the input graph is very large, it seems that your algorithm will try to load everything from train data into memory. Obviously OOM will occur.

Any suggestion?

tkipf commented 7 years ago

Hi, thanks- this is indeed a valid concern. The issue can be somewhat resolved without resorting to distributed training by simply subsampling local neighborhoods. This technique was explored in a recent paper (https://arxiv.org/abs/1706.02216) and often works well in practice.

All things considered, the algorithm does permit a distributed implementation. The communication overhead might however be a significant bottleneck in this case. In the end, it might be worth the trade-off in case the subsampling scheme degrades performance too much.

chunyang-wen commented 7 years ago

Thanks. I wonder whether you have noticed this work (Discriminative Embeddings of Latent Variable Models for Structured Data:https://arxiv.org/abs/1603.05629) .

On the same dataset cora, it achieves accuracy about 85% for test set.

It has the same idea as hash algorithm to embed nodes but uses a different approach: graph models.

tkipf commented 7 years ago

Same dataset splits (train/val/test)?

On Wed, Sep 6, 2017 at 12:56 AM Chunyang Wen notifications@github.com wrote:

Thanks. I wonder whether you have noticed this work (Discriminative Embeddings of Latent Variable Models for Structured Data: https://arxiv.org/abs/1603.05629) .

On the same dataset cora, it achieves accuracy about 85% for test set.

It has the same idea as hash algorithm to embed nodes but uses a different approach: graph models.

  • Mean field
  • Loopy belief propagation

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/tkipf/gcn/issues/15#issuecomment-327405775, or mute the thread https://github.com/notifications/unsubscribe-auth/AHAcYPmlf58rWRO2NtZ008XDAvH8haqLks5sflA4gaJpZM4PNwGO .

chunyang-wen commented 7 years ago

Yeah, maybe due to different ways of split.

bapriddy commented 5 years ago

The following paper might address the issue discussed above GCNN.

https://arxiv.org/pdf/1801.10247.pdf