A Todo List for the Sparse Feature (CPU)

Operator:

[x] sum(csr) @anirudh2290
[x] mean(csr) @anirudh2290
[x] sparse embedding(row_sparse gradient) @eric-haibin-lin
[ ] concat(csr, csr)
[x] split_axis(csr) @ZiyueHuang
[ ] eye(csr) @ZiyueHuang
[x] support custom operators with scipy @anirudh2290
[ ] unary operators for csr (arcsin, arcsinh, arctan, arctanh, ceil, expm1, floor, log1p, power, rint, sign ,sin, sinh, sqrt, tan, tanh, trunc) - most of them can just reuse the template code implemented for row_sparse
[ ] performance optimization for dot(csr.T, dense) using parallel scan
[x] _div_scalar(csr) @ZiyueHuang
[ ] NCE loss @eric-haibin-lin
[ ] dot(csr.T, csr) = row_sparse
[x] dot(dense, csr) = csr @anirudh2290
[ ] elemwise_add(csr, dense)
[ ] broadcast_add(csr) & broadcast_sub(csr)
[ ] contrib.sparse_broadcast_mul(csr) / contrib.sparse_broadcast_div(csr)
[x] _set_value(row_sparse) & setitem(int) for row_sparse
[x] norm(row_sparse)

NDArray:

[x] convenient constructor (csr) with (data, row_idx, col_idx) @anirudh2290
[x] check_format(csr & row_sparse): checks the validity of a sparse ndarray @ZiyueHuang
[x] as_scipy(csr) @anirudh2290
[ ] getitem & setitem for csr[int]
[ ] getitem & setitem for csr[py_slice]

Iterator:

Module API:

[x] expose group2ctx to Module API to enable part of the symbols trained on CPU while others are trained on GPU @ZiyueHuang

Example:

GPU Support:

[x] local kvstore push/row_sparse_pull with device comm for sparse weight of size ~ 1GB @ZiyueHuang
[x] an example which uses group2ctx for one part of the model on CPU, the other on GPU
[x] sparse embedding (row_sparse) @ZiyueHuang

Gluon Support

apache / mxnet