cuSparseLT question

[yuslepukhin]

In cuSparseLT documentation and in samples published here, 2:4 matrix is always matrix A. Though the API allows it, I am not seeing any statement if matrix B could also be a sparse matrix. Can they both be sparse?

[fbusato]

Currently, cuSPARSELt with 2:4 sparsity ratio allows only one sparse matrix A, while B and C are dense. Maybe in the future, we will support other operations between sparse matrices in the library. Please note that this does not allow Sparse Tensor Core acceleration and the performance would be much lower.

[yuslepukhin]

@fbusato Thank you for the clarification.

Please note that this does not allow Sparse Tensor Core acceleration and the performance would be much lower.

Does the above statement mean, that if both arguments are sparse, it would not allow for Sparse Tensor Core acceleration?

[fbusato]

yes, correct. If you are interested in sparse x sparse = sparse computation, you can take a look at cuSPARSE SpGEMM

[yuslepukhin]

I am more interested in:

• exceeding cuBlas dense performance
• having flexibility to have B sparse
• 2:4 format that is essentially dense which allows me to fallback to cuBlas dense computation in cases when A100 is not available
• And having windows version for the library.

[fbusato]

exceeding cuBlas dense performance

It is possible only with cuSPARSELt 2:4 sparsity computation or cuSPARSE SpMM with very high sparsity. SpGEMM is not competitive

having flexibility to have B sparse

Only possible with cuSPARSE SpGEMM

2:4 format that is essentially dense which allows me to fallback to cuBlas dense computation in cases when A100 is not available

We are considering this feature but it is not a priority right now

And having windows version for the library.

This is planned soon (next release)

[yuslepukhin]

2:4 format that is essentially dense which allows me to fallback to cuBlas dense computation in cases when A100 is not available

We are considering this feature but it is not a priority right now This should be possible today, correct? If my code checks properties and A100 is not there, I could simply feed 2:4 data to cuBlas, right?

[fbusato]

no, cuBLAS supports only dense matrices. You can simply pass the matrix to cuBLAS APIs before the compression

[yuslepukhin]

no, cuBLAS supports only dense matrices. You can simply pass the matrix to cuBLAS APIs before the compression

Right, we are on the same page.

[Achazwl]

Currently, cuSPARSELt with 2:4 sparsity ratio allows only one sparse matrix A, while B and C are dense. Maybe in the future, we will support other operations between sparse matrices in the library. Please note that this does not allow Sparse Tensor Core acceleration and the performance would be much lower.

Does cuSPARSELt allows matrix B to be sparse now, i.e. dense A * sparse B + dense C = dense output?

[fbusato]

yes, you can take a look at the documentation https://docs.nvidia.com/cuda/cusparselt/functions.html#matmul-descriptor-functions. This is computed starting from sparse A and playing with the layouts of the dense matrices.

[Achazwl]

Thanks. Would I need to upgrade from cusparseLt 0.0.1 to a newer version (0.1.0, 0.2.0, or 0.3.0)? From which version cusparseLt supports matrix B to be sparse? I have a low Cuda version (11.0) that can only run version 0.0.1, I follow the code of your example, which can do sparse A dense B, and I have used my own matrix to verify its correctness. However I can't do denseA sparseB by simply changing the matrix descriptor and applying SPMMACompress to matrix B instead of A. Do I need to use the new API cusparseLtSpMMACompress2 added in the newer version, which can specify the isSparseA argument and set it to false? It would be appreciated if you could give an additional complete code example to support B to becoming sparse!

[fbusato]

I definitely suggest you to switch to the latest version. SparseB is supported from 0.2.0. You don't need to use cusparseLtSpMMACompress2. If you still have issues after switching to the latest version, we can think to provide an example.

[Achazwl]

SparseB is supported from 0.2.0.

Wow, It works by simply changing the matrix descriptor and applying SPMMACompress to matrix B instead of A.

In addition, I am developing applications based on the cusparseLt's dynamic link library. I need to judge whether matrix B can be sparse according to the version number of cusparseLt. While the version information of cusparseLt is only reflected in #define macro, which will no go into dynamic link library; unlike cublasLt, there is a function cublasLtGetVersion() that still exists after compilation. Will similar interfaces be added to the subsequent version of cusparseLt?

[fbusato]

Thanks for the suggestion. We will add this API in the next release.

[pna961]

CHECK_CUSPARSE( cusparseLtMatmulDescriptorInit( handle.get(), &matmul, opA, opB, &matA, &matB, &matC, &matC, compute_type) ) Cry for help：when I use cuSPARSELt API to do 2:4 SPGEMM, the cusparseLtMatmulDescriptorInit always has errors as follows:On entry to cusparseLtMatmulDescriptorInit() parameter number 9 (computeType) had an illegal value: (cusparseComputeType) UNKNOWN=(cusparseComputeType) 2. Strangely, when compute_type=CUSPARSE_COMPUTE_32F and no matter what type of A, B or C is, it's always the same errors. I have already updated newest cuSPARSELt version(0.6.1).