Big push to tensor autograd

[UlisseMini]

• [ ] matmul autograd
  • [x] Explicit gradient updating to work with tensor autodiff
  • [ ] Rewrite matmul so it uses view(m,1) for vectors instead of branching
  • [ ] Implement view
    • [x] Refactor to keeping a single array for tensor data (like pytorch storage)
    • [ ] Copy pytorch stride
    • [x] Create storage with flattened data array in Tensor.new
    • [x] Track dimensions of data array and stride
    • [x] Reimplement __index and __newindex in terms of stride
    • [ ] Add multidimensional stride tests
    • [ ] Implement view in terms of changing stride
  • [ ] Test autograd through matmul
• [ ] Array datatype? (for storage, size and stride. just a table plus pretty printing) bad idea, write pretty printer instead of overwriting __tostring.
• [x] Have __index always return a tensor, combine branches into a single statement reverted: will do inheritance instead
• [ ] Move tensor operations to a separate file where they operate on pure tables (or make tensor behave so much like a table that I can use normal table functions on it)
  • [ ] Move operations/de-tensorify to ops.lua
  • [ ] Generate tensor methods automatically from pure table functions, (convert ret to tensor, etc.)

[UlisseMini]

So I originally wanted to just do autograd through matmul, and that's relatively easy. but now I'm focused on refactoring and improving code quality. Maybe I should merge this with master, I do have the tests passing after all

[UlisseMini]

Turns out the whole premise of using storage instead of nested arrays was stupid, yay!

You can totally backprop through ops like transpose tinygrad does it, the implementation complexity of storage isn't worth it, combined with the bugs that will naturally come up from all the pointers and mutation if I add view.

Trying to copy pytorch is a bad idea, (a) its written in c++ and (b) it aims to be production ready, not have a simple implementation

[UlisseMini]

Why am I doing everything on a branch again? lol