Closed Jack47 closed 3 years ago
For large tensors, we compute the forward and quantize activations one micro-batch by one micro-batch, so we call this "pipeline". This is used to reduce the temporary workspace memory and reduce memory fragmentation.
If the tensor size is larger than pipeline_threshold
, we will apply this optimization.
Sounds interesting, thanks for clear explain, it helps a lot.
I want to figure out what does pipeline and pipeline_threshold means in actnn. Din't find examples in test or readme, so may you guys give some examples or explain it a bit? Thanks![image](https://user-images.githubusercontent.com/1720972/125653804-1bccea08-ee2a-4d10-bf3d-99563ac924e7.png)
PS: I'm currently reading actnn source code and learned a lot from it , my chinese notes here.