hikettei / cl-waffe2

[Experimental] Graph and Tensor Abstraction for Deep Learning all in Common Lisp
https://hikettei.github.io/cl-waffe2/
MIT License
124 stars 5 forks source link

[Enhancement] Added define-static-node #25

Closed hikettei closed 1 year ago

hikettei commented 1 year ago

Introducing define-static-node macro: Users do not need to pay attention for the computation nodes are differentiable? or continuous? because no lazy evaluation is here.

(define-static-node (Static-Sin (self)
             :where (A[~] -> OUT[~])
             :save-for-backward-names (x-input)
             :forward ((self x)
                       (print "Hi :) the operation sin is executed.")
                       (with-setting-save4bw ((x-input x))
                            (proceed (!sin x))))
             :backward ((self dout)
                             (with-reading-save4bw ((x x-input))
                                   (values (proceed (!mul (!cos x) dout)))))))

Calling Static-Sin but the print function is still not yet called.

(call (Static-Sin) (randn `(3 3)))
{CPUTENSOR[float] :shape (3 3) :named ChainTMP22057 
  :vec-state [maybe-not-computed]
  <<Not-Embodied (3 3) Tensor>>
  :facet :input
  :requires-grad NIL
  :backward <Node: STATIC-SIN-T (A[~] -> OUT[~])>}

The moment someone accept the computation node and invoked it, print is called.

(proceed *)

Hi :) the operation sin is executed.
{CPUTENSOR[float] :shape (3 3) :named ChainTMP22130 
  :vec-state [computed]
  ((-0.44811794 -0.8374244  -0.9075781)
   (-0.9591228  0.58454794  0.9774129)
   (-0.8381093  -0.36447936 0.9476587))
  :facet :input
  :requires-grad NIL
  :backward <Node: PROCEEDNODE-T (A[~] -> A[~])>}