search

dmlc / dgl

Python package built to ease deep learning on graph, on top of existing DL frameworks.
http://dgl.ai
Apache License 2.0
13.57k stars 3.02k forks source link

[GraphBolt][CUDA] Implement Cooperative Minibatching from arXiv:2310.12403 #7273

Open mfbalin opened 7 months ago

mfbalin commented 7 months ago

Towards implementing arXiv:2310.12403.

Background on Cooperative Minibatching's history:

Aug 2022: #4337 was the first publicly available full implementation of the idea. January 2023: the first submission of the paper was made to ICML, that version can be found here, Cooperative Minibatching ICML submission. March 2023: Subsequent work by @sandeep06011991 in March 2023 explored a similar but more limited version, sampling being performed on the CPU and then later splitting to the available GPUs. June 2024, GSplit authors made an exact replica of our work when it comes to Cooperative Sampling, Feature Loading and Training on GPUs: https://arxiv.org/abs/2303.13775v2 without giving our paper any credit when we were busy submitting our work citing their work as concurrent work over multiple submission cycles on a goodwill basis. This was after discussing and deciding to cite each other as concurrent work with the first author (@sandeep06011991) of the GSplit paper 1 year ago.

mfbalin commented 7 months ago

@nv-dlasalle GraphBolt will have full support for Cooperative Minibatching, it is going to be even more optimized than in our paper, it will provide speedups even for cheap GNN models such as GraphSAGE forward/backward passes.