search

NVIDIA / CUDALibrarySamples

CUDA Library Samples
Other
1.5k stars 311 forks source link

[cuSPARSELT] Documentation on 2:4 sparse packed format created by cusparseLtSpMMACompress? #167

Closed cj831880 closed 8 months ago

cj831880 commented 8 months ago

Where can we find documentation on the 2:4 sparse packed format created by cusparseLtSpMMACompress? We tried to consult Section 3.1 in https://arxiv.org/abs/2104.08378 but the information on the format is, well, sparse. Thank you.

fbusato commented 8 months ago

Hi @cj831880, unfortunately, the internal layout of the metadata is only internal because it can change at any time. If you really need to interact with metadata, my suggestion is to take a look at CUTLASS and the reordering routine. https://github.com/NVIDIA/cutlass/blob/b5d8a5d9ccb435268e2215144dca162b0fc6a826/tools/util/include/cutlass/util/host_reorder.h#L83

cj831880 commented 8 months ago

Thank you @fbusato. Unfortunately CUTLASS performance is terrible for our use case, so we need to leverage the internal 2:4 sparse format used by cusparseLtSpMMACompress to reuse computation and improve performance. Any chance we could get documentation on the sparse format regardless? Thank you and happy new year!

fbusato commented 8 months ago

Got it. We cannot reveal the internal format at this time. You can even try to combine the CUTLASS reordering routine and cusparselt, but compatibility is not guaranteed. If you really need it, you can try to reach me at my work email and I will bring the issue to the PM.