The second option may allow easier autodiff implementation since it's the syntax used in this paper.
I'm not clear on the mechanics of how to construct the graph: whether a tensor would be an Expr instead, and the only global aspect would be the counter variable (like we had before but without the need to merge trees). It might be that this approach would allow us to use ST Nat instead of linear mutable arrays, which would remove a dependency. I imagine that would be as fast, but we'd have think carefully about that.
Currently we capture the graph as a mapping of node positions to nodes:
SortedMap Nat Expr
. Then tensors are justNat
indices into this graph.The alternative approach is to include a "let" construct in
Expr
by addingLet
andVar
nodesExample for
is
The second option may allow easier autodiff implementation since it's the syntax used in this paper.
I'm not clear on the mechanics of how to construct the graph: whether a tensor would be an
Expr
instead, and the only global aspect would be the counter variable (like we had before but without the need to merge trees). It might be that this approach would allow us to useST Nat
instead of linear mutable arrays, which would remove a dependency. I imagine that would be as fast, but we'd have think carefully about that.