hikettei / cl-waffe2

[Experimental] Graph and Tensor Abstraction for Deep Learning all in Common Lisp

https://hikettei.github.io/cl-waffe2/

MIT License

122 stars 5 forks source link

Enhancements on JITCPUBackend #141

Closed hikettei closed 8 months ago

hikettei commented 8 months ago

Changes

JITCPUTensor

[Optimize] Increased the stability of OpenMP Support
[Optimize] Tiling, Unrolling, Loop Reordering to maximize the use of L1/L2 caches
- !permute actually works 2x~3x times faster than Numpy
- This is the only case of smaller batch-size(2~10) but Conv2D works 5x times faster than PyTorch
[Optimize] Simplify the computation of strides
[Optimize] The backend is now able to detect reduction.

What's still missing?

Im2Col, Col2Im nodes are remained to be optimized:
- It is possible for me to write an optimized kernel, but it violates the concepts of cl-waffe2
- Its implementation should be realized by combining !move and !view but it beyonds cl-waffe2 due to:
  - 1. The implementation of view.lisp call-with-view.lisp do-compiled-loop.lisp should be much more polished!
  - 1. I wish !view can handle (range X Y Z) like Petalisp
  - or should I adapt Petalisp on cl-waffe2 backends?