Why did the resolution of ADC can be reduced ?

[yanghr]

Thanks for your question! This comes from the design logistics of ReRAM-based processing elements. For each column of ReRAM crossbar, an ADC is attached at the bottom to convert the accumulated current to a digital value, which corresponds to the result of a input row vector on the bitlines multiplies the column vector stored in the crossbar. The ADC resolution will be set to accomodate the largest possible output value, which is achieved when all the input bitlines are set to 1. With this being said, as our work can significantly reduce the number of zeros stored in each column of ReRAM, even if all the inputs are 1, the maximum possible output value will be significantly reduced. This will enable us to reduce ADC resolution without hurting the system performance.

[shenlinyao]

So the reason why the project does bit-level sparsity here is not to decrease the need of storage for weight or reduce the procedure of computation. It mainly aims to reduce the resolution of ADC. And how many bits of ADC can be reduced is not precisely determined by the algorithm but by the statistical data of the number of non-zero bit in each column?

[yanghr]

For common ReRAM-based NN accelerator design, the unstructured sparsity in the weight matrix do not bring much benefit on reducing computation cost, which is already extremely low thanks to the compute-in-memory nature of ReRAM crossbar. Therefore for now the major benefit of bit-slice sparsity is to potentially reduce the ADC overhead needed. In the meantime, there do exist ongoing research on designing new ReRAM architecture that can utilize unstructured sparsity to further reduce the cost. As the weights are stored as bit-slices on the crossbar, the ability to induce bit-slice sparsity would potentially lead to new path on software-hardware codesign of ReRAM-based devices.