hikettei / cl-waffe2

[Experimental] Graph and Tensor Abstraction for Deep Learning all in Common Lisp
https://hikettei.github.io/cl-waffe2/
MIT License
122 stars 5 forks source link

Enhancements on IR #91

Closed hikettei closed 10 months ago

hikettei commented 10 months ago

Userful Benchmarking Tools

I introduced a proceed-bench function which is really useful for benchmarking!

(proceed-bench (!sum (randn `(3 3))))
 Time(s) |   Instruction ( * - Beyonds the average execution time)
1.8e-5*  | <WfInst[Compiled: SCALARMUL-CPUTENSOR] : TID1409459 <= op(TID1409459(1 1) <Input>TID1409461(1))>
1.0e-6   | <WfInst[Compiled: VIEWTENSORNODE-T]    : TID1409470 <= op(TID1409470(3 3) TID1409459(1 1))>
3.0e-6   | <WfInst[Compiled: ADDNODE-CPUTENSOR]   : TID1409470 <= op(TID1409470(3 3) <Input>TID1409456(3 3))>
1.0e-6   | <WfInst[Compiled: VIEWTENSORNODE-T]    : TID1409492 <= op(TID1409492(1 1) TID1409470(3 3))>

4 Instructions | 5 Tensors

 Total Time: 2.3e-5 sec

 Instruction                           | Total time (s) | Time/Total (n-sample=1)
<WfInst[Compiled: SCALARMUL-CPUTENSOR] | 1.8e-5 | 78.26087%
<WfInst[Compiled: ADDNODE-CPUTENSOR]   | 3.0e-6 | 13.043478%
<WfInst[Compiled: VIEWTENSORNODE-T]    | 2.0e-6 | 8.695652%
{CPUTENSOR[float] :shape (1 1) -> :view (<(BROADCAST 1)> <(BROADCAST 1)>) -> :visible-shape (1 1) :named ChainTMP1409458 
  ((1.0014453))
  :facet :input
  :requires-grad NIL
  :backward NIL}

Adjusted minor changes on in-place mutation

  1. Fixed the condition
  2. Disabled when training