Closed iamgeo92 closed 1 year ago
Checkout this branch: https://github.com/liuliu/swift-diffusion/tree/liu/nhwc
Main thing is to move from NCHW layout (more efficient on CUDA) to NHWC layout (more efficient on Apple hardware).
Main thing is to move from NCHW layout (more efficient on CUDA) to NHWC layout (more efficient on Apple hardware).
For someone reading this in the future CUDA is also faster on NHWC layout 🐎 https://docs.nvidia.com/deeplearning/performance/dl-performance-convolutional/index.html#tensor-layout
For someone reading this in the future CUDA is also faster on NHWC layout 🐎 https://docs.nvidia.com/deeplearning/performance/dl-performance-convolutional/index.html#tensor-layout
That's good to know! I should do more testing, probably because I wrongly did NHWC for the weights layout and that doesn't activate winograd properly in conv layer.
Maple-diffusion ( the MPS inplementation) is like 1-1.3s per iteration on 8GB M1 mac. But your implementation takes 90 secs for 30 steps.
I was wondering what is lacking in this codebase? and how can we get speed similar to Draw Things app for M1.
Thanks