search

UDC-GAC / venom

A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
Apache License 2.0
27 stars 4 forks source link

float tensor pass into int* #6

Open ENXING opened 3 weeks ago

ENXING commented 3 weeks ago

Code https://github.com/UDC-GAC/venom/blob/8ddaf38ef918d4aedfea12696fed60348c5ccd10/end2end/gpt2_pytorch.py#L74

input tensor is float, so mask is float type. but "SrNMTensor" initialization with parameters "mask" and call to_sparse_sr_nm

https://github.com/UDC-GAC/venom/blob/8ddaf38ef918d4aedfea12696fed60348c5ccd10/end2end/grouped_nmv_tensor.py#L692

And it will call c function with mask int type

https://github.com/UDC-GAC/venom/blob/8ddaf38ef918d4aedfea12696fed60348c5ccd10/end2end/grouped_nmv_tensor.py#L529

this is a bug.

LopezCastroRoberto commented 3 weeks ago

Thanks for reporting this! This may be a typo that was introduced in code cleaning and refactoring. I will take a look at it.

ENXING commented 3 weeks ago

@LopezCastroRoberto Thanks. I would appreciate if you can add a validation between sparse matrix multiplication in torch and calling your library in the sparse_dense_mul_dispatch. I find they don't match.

Also, Do I understand correctly: Weight with mask is the equivalent as your sparse matrix representation? I feel confused why there is this line if that is true, it uses ha_column to set column index instead of mask matrix alone. I expect HA_column and ha_value, ha_meta are output matrix, but here HA_column is input. https://github.com/UDC-GAC/venom/blob/8ddaf38ef918d4aedfea12696fed60348c5ccd10/end2end/grouped_nmv_tensor.py#L555 https://github.com/UDC-GAC/venom/blob/8ddaf38ef918d4aedfea12696fed60348c5ccd10/end2end/grouped_nmv_tensor.py#L556

p81sunshine commented 1 week ago

Hi, ENXING. I have been carefully reviewing this code recently. Do you have any contact information? Maybe we can discuss it in detail.