[Roadmap] v0.3 release checklist

[jermainewang]

Here is the v0.3 release plan. The tentative release date is 06/07.

[Feature] Kernel support

Kernels are critical for our system performance. The next release will include all the basic building blocks and APIs for future extensions.

• [x] Basic fused kernel support (#519 )
• [x] New builtins (@jermainewang @ylfdq1118 ) [05/29]
  • src_op_edge, src_op_dst, edge_op_dst, dst_op_edge, etc.
• [x] MXNet fused kernel support (#541 @ylfdq1118 ) [05/29]
• [x] Sparse softmax module & GroupEdgeApply [06/01]
• [x] Pip/conda installation (@BarclayII @jermainewang ) [06/06]

[Feature] Giant graph support

Release a demo about how to train GNNs on giant graphs that cannot be hosted in a single GPU memory. This includes:

• [x] Sampler and Nodeflow tutorial (#522 @zheng-da )
• [x] Shared-memory graph store and distributed sampler tutorial (@zheng-da @aksnzhy)
• [x] Distributed sampler
  • [x] RPC and network communication (#465 @aksnzhy )
  • [x] Distributed sampler example (#564 #562 @aksnzhy )
• [x] GraphStore APIs (#468 @zheng-da )
• [x] End-to-end single-machine shared-memory based giant graph example. (@zheng-da )

[Enhancement] NN module

• [x] Efficient EdgeSoftmax using new builtins
• [ ] GAT

[Enhancement] graph structure

• [x] Refactor immutable graph to have better CSR/COO, context and datatype support (#543 @jermainewang )
• [x] Batching immutable graphs (#539 @VoVAllen )
• [x] Random walk APIs (#453 @BarclayII )
• [x] Support edge/node removal (Issue #467 @yzh119 PR #599 )
• [x] Support convenient DGLGraph.to(device) (Issue #503 @HQ01 PR #600 )
• [x] dgl.to_simple_graph API to remove multi-edges (#587 @jermainewang )
• [x] dgl.to_bidirected API to transform the graph to bidrectional. (#598 @mufeili )

Model Examples

• [x] APPNP (#480 @aymenwah )
• [x] GIN (#471 @kitaev-chen )
• [x] PinSage (#453 @BarclayII )

Bug fix

• [x] NodeFlow bugs (#531 @zheng-da )
• [x] NumPy bug (Issue #538 PR #580 @jermainewang )

Features postponed to v0.4

[Feature] DGL graph data format As we want to include more popular graph dataset in DGL, it is time to decouple dataset with DGL repo.

• Design a unified graph data format for DGL data loader. The data format should be (1) efficient for read/write; (2) easy for partitioning
• Fast data loader in C++.
• Convert our curated dataset to such format.
• More dataset?

[Feature] Bipartite (k-partite) graph API Bipartite graph is popular in recommendation setting. We heard many requests for this.

[Enhancement] Improve IR & scheduling system

• Improve pull by using slicing on CSR graph.

[yzh119]

Some additional items:

• Kernel Support
  • sparse_softmax
• Convenient RandomWalk api
  • to support algorithms like personalized page rank.
• Model Examples
  • relational network/recurrent relational network/relational deep reinforcement learning(however this one is much more difficult).
  • these models are good showcases as using graph to modeling text/image.

[yzh119]

For API, I suggest providing more flexible message function to handle different type of edges. For example, In tree-lstm we are supposed to send different messages for left branch and right branch.

[aksnzhy]

We plan to implement a light-weight kvstore instead of using MXNet KVStore for DGL. Do we consider add this item to 0.3 version?

[zheng-da]

• kernel support:

  • scatter mean
  • generalized spmv: min/max/mean for reduce.

• NN module:

  • all NN module should have a MXNet version.

• dataset:

  • reddit: we construct multiple versions at different scales and different ways of construction (mainly how to build edges between nodes).
  • yelp

• Scheduling improvement:

  • we should improve pull. (It's commonly used by control-variate sampling and we run pull on the giant graph.)

• Giant graph:

  • DGLStore for shared memory (optimized for large-memory multi-processor machines).
  • More compact immutable graph structure to reduce memory consumption for giant graphs.

• Model examples:

  • Genipath

If time permits, we might add GPU samplers to accelerate mini-batch training in a single GPU.

[zheng-da]

We plan to implement a light-weight kvstore instead of using MXNet KVStore for DGL. Do we consider add this item to 0.3 version?

0.3 version will be a fast release. How about we plan it for 0.4 release?

[aksnzhy]

We plan to implement a light-weight kvstore instead of using MXNet KVStore for DGL. Do we consider add this item to 0.3 version?

0.3 version will be a fast release. How about we plan it for 0.4 release?

No problem.

[qzshadow]

Looking forward to supports for giant graphs!

[alexvpickering]

A few models I'd like to see:

• Graph Isomorphism Network
• LanczosNet

[jermainewang]

Hi @alexvpickering , will look into it. One question, do you expect to see them as layers in NN packages or complete examples with data loading/training/testing ?

[alexvpickering]

Thanks @jermainewang! I was thinking examples like found in e.g. dgl/examples/pytorch/sgc.

[ghost]

will some model in Euler like LsHNE, LasGNN, ScalableGCN implement in the feature? These algorithm are like Pinsage Application in production environment and support Heterogeneous Network.

[zheng-da]

@HuangZhanPeng We will work on heterogeneous graphs in the 0.4 release. We'll evaluate these models.

[gasteigerjo]

It would be great if you could also implement APPNP! It's quite simple and performed best in PyTorch Geometric's benchmark, so people could clearly benefit here as well. :)

[mufeili]

Thank you for the suggestion @klicperajo . I think we've already had one implementation for this work :), though we may need to tune it more carefully to achieve a better performance. See dgl APPNP example.

[gasteigerjo]

Oh, nice! I didn't notice since it's not in the summary table. Yes, there are a couple of details to get the last tens of percent accuracy. Should be all in the paper, though. :)

[jermainewang]

Just updated the roadmap with a checklist. Our tentative date for this release is 06/07.

For all committers @zheng-da @szha @BarclayII @VoVAllen @ylfdq1118 @yzh119 @GaiYu0 @mufeili @aksnzhy @zzhang-cn @ZiyueHuang , please vote with :+1: if you agree with this plan.

[szha]

did you mean 6/7?

[VoVAllen]

@yzh119 will be in charge of edge/node removal

[mufeili]

@jermainewang What do you need for issues under NN module?

[yzh119]

One question: do we need to rebuild node/edge index when calling node/edge removal? For example:

import dgl
import torch as th
g = dgl.DGLGraph()
g.add_nodes(5)
g.ndatas['x'] = th.rand(5, 3)
g.del_nodes([2, 3])
print(g.nodes())

What should be the output? tensor([0, 1, 4])?

[yzh119]

I must say, if the node index and edge index have to be rebuilt after node/edge removal, the behavior of these operations would be VERY CONFUSING. I can't see any scenario that these operations would make any sense.

[VoVAllen]

I think we need to return the removed edges, and let user handle the mapping between original index and modified index. (or provide utils function)

[yzh119]

@VoVAllen , I don't see any benefits of doing this compared to creating a subgraph of the current graph.

[jermainewang]

@jermainewang What do you need for issues under NN module?

(1) EdgeSoftmax needs to be re-implemented using the new builtins; (2) GAT as an NN module. Both depend on the kernel branch to be fully merged.

@mufeili do you have time take over these two items?

• dgl.to_simple_graph API to remove multi-edges
• dgl.to_bidirected API to transform the graph to bidrectional. (Issue #419)

[mufeili]

@jermainewang I may need to wait till weekend for starting implementation. If that's good for you, I'll take over them.

[jermainewang]

I've taken to_simple_graph. The rest is yours.

[jermainewang]

v0.3 has been released. Thanks everyone!