Team members: <E24076459蕭又瑜
><E24071069陳志瑜
><E24073037鍾震
>
CNN has been widely used in many image-related machine learning algorithms due to its high accuracy for image recognition. Convolution and fully-connected layers are two essential components for CNN. Our goal is to design a simple TPU to accelerate them.
ref: https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/
k
input values in the input feature map (or say, number of neurons) and n
output neurons, so there are totally k * n
weights
Height
by Width
by Channel
![]() |
![]() |
---|
Here we use a function called im2col
im2col
stands for image-to-columnim2col
convert the 3D image into a 2D array that we can treat like a matrix
Also, we have to flatten the kernels, where k
is the number of values in each patch so it is kernel Width
Height
Depth
The resulting matrix will be number of Patches high and Number of Kerenels wide
k
n
times.m
times"If a structrue can truly be decomposed into a few types of simple substructures or building blocks, which are used repetitively with simple interfaces, great savings can be achieved." - H.T. Kung, ref: http://www.eecs.harvard.edu/~htk/publication/1982-kung-why-systolic-architecture.pdf
In this project, the target model is a simple CNN for MNIST dataset, which is trained with tensoflow framework. Our goal is to design a TPU with high compatibility, so RGB-input model is also acceptable. Here is our model summary.
Two functions are designed to do the transformation between matrix and image:
Although Convolution and Fully connect will be accelerated on TPU in PL, PS still have to deal all the other functions like: activation, pooling, softmax… Also, Keras framework is not supported on ARM CPU. To eliminate the dependency, model's weights are extracted and re-built it on ZYNQ processor without importing tensorflow library. We also have to use self made CONV and FC function to do algorithm simulation, ensuring the correctness before HW design. Also, all of them have an fixed-point version.
It’s easier to consider floating point number as fixed point for hardware design. However, a bad quantization might decrease accuracy seriously. The follwing shows Some experiments with different fraction bit length for our model.
We found that setting fraction bit length to 4~8 would yield a better result. Finally, 8 is chosen.
A simple model with only one convolution layer is used to visualize our quantization results.
Since our matrices size are quite large (at least 16kB each), the heavy data transmission may become the bottleneck.
Our solution:
AXI Central DMA (CDMA) provides high-bandwidth Direct Memory Access (DMA) between a memory-mapped source address and a memory-mapped destination address using the AXI4 protocol. In our project it's used to connect ZYNQ Processor DDR3 High Peformance Pin and Block RAM. Also, AXI Interconnection module is introdced to help us deal with Master/Slave connection.
The following figures show our memory map layout for 3 I/O buffer (global_buffer) of TPU.
After computation below, 3 of 64KB buffers are implemented.
Note that each word length in PL is 128-bits.
Total size of block memory on PYNQ is 630KB: 630KB x 11.43 % = 72KB / IO buffer The difference between 64KB and 72KB is due to ECC.
“CDMA + TDP BRAM” version
“SDP Asymmetric BRAM” version
This simulation shows that our TPU works fine.
C++ code generated pattern :
HDL waveform:
Unfortunately, on PYNQ, results from our system cannot match the one from SW.
We find that the output after convolution is all zero.