[API Enhancement] Project Simplification for those who is new to Common Lisp

[hikettei]

• TODO
  • [ ] Step-by-step tutorial for those who aren't familiar with Deep Learning.
  • [ ] Tutorial written in Jupyter Notebook
  • [ ] Translate tutorial_jp.lisp into English
  • [ ] Visualized Explanation of cl-waffe2 (should i make a slide?)
  • [x] I think deftrainer usage should be much more documented, because it is unique and complicated.

[hikettei]

We need to simplify the list of instructions needed for backend extensions:

Cheat sheet for backend extension

A backend which is completely separated from the standard implementation can be established by implementing all of the following instructions:

;; Arithmetic
AddNode SubNode MulNode DivNode
ScalarAdd ScalarSub ScalarMul ScalarDiv
InverseNode

;; [TODO] should be reduced to =>
AddNode SubNode MulNode DivNode

;; Matrix Transformation
MoveTensorNode

MoveTensorNode but JITxxxBackend dedicated to arithmetic ops for fast permute computation.

;; Mathematical Functions
SinNode CosNode TanNode ...

;; Logical Operations

;; Neural Network specialized instructions

Im2ColNode
Col2ImNode

To put it bluntly, instructions needs to be reimplemented are divided to three types:

• CFFI Interface Binding (generated by call-with-view)

  • BLAS APIS (axpy, gemm etc...)
  • CFFI Interface like cl-bmas

• JIT Backend Extension just following the instruction of solve-loop-order

  • Example: JITCPUTensor
  • almost 300 line JIT compiler works enough, just used to optimize the layout of memory. Example: JITCPUTensor

• Device-specific (Neural Network) operations

  • e.g.: Im2Col, Col2Im ReLUNode(optional)

Our goal is: just 1000 line CUDA Extension