Function request: Sparse matrix inverse

[lith0613]

I want to calculate the inverse of a sparse matrix, how can I do this ?

for example

vers_n=200000
ind_i = torch.Tensor(list(range(vers_n))).long().view(1,-1)
ind_j = torch.Tensor(list(range(vers_n))).long().view(1,-1)
indices = torch.cat((ind_i, ind_j), dim=0)
values = torch.ones(vers_n)
I = torch.sparse.FloatTensor(indices, values, (vers_n, vers_n))
print(I.inverse())

then will get the error:

RuntimeError: Could not run 'aten::_inverse_helper' with arguments from the 'SparseCPUTensorId' backend. 'aten::_inverse_helper' is only availabl

Maybe a possible solution is turn to dense and calculate the inverse, but if I turn to dense, it will be out of memory and exit 137.

cc @jianyuh @nikitaved @pearu @mruberry @heitorschueroff @walterddr @IvanYashchuk @xwang233 @Lezcano @cpuhrsch @aocsa

[mruberry]

Hey @lith0613, unfortunately sparse tensors don't currently support inverse computations, so you're correct that you'd need to convert the tensor into a dense one first to take its inverse. SciPy has a function for computing the inverse of a sparse matrix that you may be able to use: https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.inv.html.

I've changed the title of this issue to be a request for an implementation of sparse inverse in PyTorch.

[AHolliday]

Just commenting to signal my desire for this feature as well. I'm working on a graph neural network project and have to use adjacency matrices of very large but sparsely-connected graphs. My GPU can't handle the memory requirements of doing everything with dense tensors, but I need to compute matrix inverses.

[IvanYashchuk]

In general, the inverse of a sparse matrix is not sparse so it will likely hit the memory limit even if we had such functionality. If this inverse matrix is later multiplied with a vector or matrix, a faster and better way to compute this would be by treating it like a solution to a linear system, this can be efficiently solved using Krylov iterative methods, like GMRES.

Even though it's generally not recommended to compute inverses explicitly, we are likely to provide such functionality when we have QR- or LU-based solving of linear systems (using cuSolverSp library).

[AlphaBetaGamma96]

Is there any update on this issue?

[marios1861]

I think this is a really big limitation and needs higher priority.