search

hikettei / cl-waffe2

[Experimental] Graph and Tensor Abstraction for Deep Learning all in Common Lisp
https://hikettei.github.io/cl-waffe2/
MIT License
122 stars 5 forks source link

[Refactor] APIs for Network Construction #99

Closed hikettei closed 10 months ago

hikettei commented 10 months ago

Overview of Computation Node in cl-waffe2

[AbstractNode] The fundamental unit which binds forward/backward propagations.
 defnode - Declares a general definition of AbstractNode
      L define-impl        implements AbstractNode. Its forward definition is given as a macro (to inline/call-with-view), later (compile nil body) is called, and cached.
      L define-impl-op  Implements as a lambda function.

define-op = defnode + define-impl-op

[Composite] Bundles several AbstractNodes, defined by defmodel macro.
  defmodel - Defines a new Composite
     L defmodel-as Redefining the existing Composite as a function or AbstractNode to reduce compiling time, to use cl-waffe2 as a define-by-run library.

Accordingly, these macros will be deleted in the future release: define-static-node, define-composite-function.

hikettei commented 10 months ago

これしたらdefpathとFusionOpsとADの数値的安定性向上に取り組む (<- save_for_backwardのためのMoveTensorNodeを削除したのち、FuseOps) + ドキュメント更新 + proceedで実行してもメモリの管理ちゃんとできるようにする + RNN動かす + define-by-run mode

hikettei commented 10 months ago

Changes are merged into develop at #100