[NOT FOR MERGE] Reconfigurable Polyhedral Scheduler

[hikettei]

• [ ] Recreate this PR w/ necessary changes
• Implement Configurable Polyhedral Compiler (cost functions are configurable by Device)
  • [ ] SERIALIZE=1 in default
  • [ ] Refactor: Delete existing schedule, replace w/ polyhedral-group.lisp
  • [ ] Fuse ConvND < 1 Kernels by using polyhedral-group
  • [ ] TILE=1
  • [ ] Transformer Scheduling w/ proper group partition and reasonable time

    [Lowerer] (Scheduled-Items)
    |
    [Post-MultiExpr] (doing pre loop fusion)
    |
    [Polyhedral-Group] (TILE, affine fusion, etc)
    |
    [Post-MultiExpr here?]
    |
    Final

[hikettei]

• minimal repro for slow transformer kernel (outermost coincidence is not what I need)

  (caten (call (feedforward 10 10) (!add (call (Embedding 10 10) (make-tensor `(3 3))) (call (Embedding 10 10) (make-tensor `(3 3))))))

[hikettei]

memo: in transposed gemm fusion and convnd scheduling

  for (int c0 = 0; c0 <= 9; c0 += 1)
    for (int c1 = 0; c1 <= 29; c1 += 1) {
      T1(c0, c1);
      for (int c2 = 0; c2 <= 19; c2 += 1)
       // T0 should be moved to here by permuting
        T2(c0, c1, c2);
      T3(c0, c1);
    }
  for (int c0 = 0; c0 <= 19; c0 += 1)
    for (int c1 = 0; c1 <= 29; c1 += 1)
      T0(c0, c1);  

[hikettei]

• Step1
• Matmul+Transpose, ConvND Fusion in transform.lisp (Supported by ISL)
• Step2
• Proper Kernel Partition for larger graph like Transforer
• Step3
• Apply Loop Collapse/Tile to the final kernel
• [ ] TC-Like notation
• [ ] Support Loop Fusion at ./ajit/polyhedral-group.lisp
• [ ] convnd