Shrimp-AI / shrimpgrad

Yet another tensor library
GNU General Public License v3.0
10 stars 0 forks source link

Add ShrimpJit decorator #6

Closed kvkenyon closed 2 months ago

kvkenyon commented 3 months ago

Once you define the initial thunk graph via constructing a model via chained tensor ops, the thunk graph doesn't need to be regenerated every time we execute the same forward pass (and backward pass). Nor do the kernels. For inference it doesn't really matter since inputs are always different and model parameters are fixed (and there is no backward pass), but for training we update model parameters and repeatedly run forward/backward passes with the same kernels. Right now they all get re-generated when Tensor.realize is called. Ideally the optimizer updates the weights, and we just recompute the pass with the already generated kernels. The update of the weights only affect the buffers anyway.

kvkenyon commented 3 months ago

Starting to add jit today

kvkenyon commented 2 months ago

JIT now works for training models, we can close this issue for now.