Roadmap

[UlisseMini]

A list of things to do before this is usable in practice. Main goals:

1. Be as simple, short and readable as possible
2. Be decoupled, be able to use each module independently of the others

1.0 TODO

• [ ] Rock solid value based autodiff implementation
  • [ ] Nice way to define new ops, inside of Value module such that they get automatically tested
  • [x] Test all derivatives numerically
  • [ ] Test complex autodiff programs numerically
• [ ] Delete Tensor autograd
• [ ] Make Tensor generic so it can hold Value's or number's (keep them decoupled)
• [ ] Test tensor autodiff a bunch numerically
• [ ] Examples
  • [ ] Least squares:, minimize||Ax - b||

Extra

• [ ] Higher order autodiff
• [ ] Replace math.* functions on Value to work with autodiff, ala multiple dispatch
• [ ] Don't compute derivatives of constants (save compute, don't convert to Value just to throw away the gradient)
• [ ] Linear algebra routines
  • [ ] Solve Ax = b, find the inverse
  • [ ] Find eigenvectors of a matrix

[UlisseMini]

This is outdated, I keep an offline todo list that's more up to date