hikettei / cl-waffe2

[Experimental] Graph and Tensor Abstraction for Deep Learning all in Common Lisp
https://hikettei.github.io/cl-waffe2/
MIT License
122 stars 5 forks source link

[Fix] Some specifications on Subscript DSL should be changed in the future release #87

Closed hikettei closed 9 months ago

hikettei commented 10 months ago

Current problems (as far as I know) in Subscript DSL is the following:

Difficulty in expressing complicated transmission states

       :where (Input[N C_in H_in W_in] -> Output[N C_out H_out W_out]
               where
               C_in  = in-channels
               C_out = out-channels
               ;; H_out = floor(((H_in + 2 * padding[0] - dilation[0] * (kernel_size[0] - 1) - 1) / stride[0]) + 1)
               H_out = (if (numberp H_in) ;; If H_in is a symbol, return -1 (=undetermined, later determined.)
                       (floor (+ 1 (/ (+ H_in (* 2 (car padding)) (* (- (car dilation)) (- (car kernel-size) 1)) -1)
                              (car stride))))
                       -1)
               ;; W_out = floor(((W_in + 2 * padding[1] - dilation[1] * (kernel_size[1] - 1) - 1) / stride[1]) + 1)
               W_out = (if (numberp W_in)
                       (floor (+ 1 (/ (+ W_in (* 2 (second padding)) (* (- (second dilation)) (- (second kernel-size) 1)) -1)
                              (second stride))))
                       -1))

(at https://github.com/hikettei/cl-waffe2/blob/master/source/nn/conv.lisp#L76)

Since both Convolution and Pooling have too complicated transmission states to express lazily, the :where form returns -1 (can't predict) when the result won't become an integer. This ugly behaviour should be fixed in the future release, but I don't have any idea.

The implementation is ugly

Should be refactored: https://github.com/hikettei/cl-waffe2/blob/master/source/vm/nodes/shape.lisp

Polish generated Shape-Error

(At https://github.com/hikettei/cl-waffe2/blob/master/source/vm/nodes/shape-error.lisp)

Shaping-Error should be more pinpoint, In fact, it is possible to make it easy to know where should be fixed, and at which node the error occurred.

hikettei commented 9 months ago

This issue is integrated to #132