search

NVIDIA / cutlass

CUDA Templates for Linear Algebra Subroutines
Other
5.5k stars 939 forks source link

[QST] Any support or examples of uint1_t x int1_t GEMM? #535

Closed Akimoto-Cris closed 2 years ago

Akimoto-Cris commented 2 years ago

Is b1 x b1 GEMMs all implemented by XOR that requires uint1_t x uint1_t ? What if A=uint1_t and B=int1_t ? (e.g. A is ReLU output, B is weight)

Thanks in advance.

hwu36 commented 2 years ago

Is b1 x b1 GEMMs all implemented by XOR that requires uint1_t x uint1_t ?

We supports both xor_popc and and_popc. See this one https://github.com/NVIDIA/cutlass/blob/master/include/cutlass/arch/mma_sm80.h#L2017 and this one https://github.com/NVIDIA/cutlass/blob/master/include/cutlass/arch/mma_sm80.h#L2084

What if A=uint1_t and B=int1_t ?

signed int1_t is impossible. sign bit would take the only available bit and you have no place to store the data.

Akimoto-Cris commented 2 years ago

Hi @hwu36 , I've seen works that use signed representation, e.g., Xnor-net: Imagenet classification using binary convolutional neural networks, who signed representation for both X and W and supposedly uses XNOR-Bitcount op for conv. So I think signed 1-bit W is a realistic scenario.

hwu36 commented 2 years ago

Maybe it is a question to the paper author. I don't know how to pack signed bit and data bit into 1 bit. xoring with signed bit also sounds strange to me. Maybe they used 0 to represent -1? If so, that's customized data representation.

Akimoto-Cris commented 2 years ago

Check this post https://sushscience.wordpress.com/2017/10/01/understanding-binary-neural-networks/. They do use 0 to represent -1. But I suspect current xor_popc and and_popc will work on such "bipolar binary". This project seems implement xnor_popc for that: https://github.com/Xilinx/finn/issues/31

github-actions[bot] commented 2 years ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.