search

UlisseMini / light

Autodiff in pure lua. Think pytorch but a lot slower
5 stars 0 forks source link

Closures capture state leading to inflated memory #2

Closed UlisseMini closed 3 years ago

UlisseMini commented 3 years ago

Every _backwards closures closes over surrounding state, leading to high memory usage and possible memory leaks.

Possible fixes are

  1. Don't use closures, use an op constructor like pytorch
  2. Lean all the way into closures, getting rid of the _parents list, having the closures do everything with recursion

I prefer (1) since eventually I can make graphs like this. and (2) seems like a hell to debug.

TODO:

UlisseMini commented 3 years ago

In my new rewrite I'm not using closures much, and (a) the closure closes over a pointer reference so it isn't copying anything (b) if you use the same param name the function param shadows the closed value