Open MariusDrulea opened 6 months ago
Currently the scan method is used to mark the nodes before applying the actual backpropagation in the graph. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19 We can achieve the same results without employing the this pre-marking of nodes. In this proposal the back function shall not use the scan method, but rather take care of the nodes itself, without a pre-marking. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L74
scan
back
The functions and files involved are only where the ref field is used: https://github.com/FluxML/Tracker.jl/blob/master/src/Tracker.jl#L41.
ref
The following structures/functions/algos have to be understood in order to remove scan:
back.jl
Note the function back(g::Grads, x::Tracked, Δ) is used for higher-gradients and we should not focus on this use-case in this proposal.
@MariusDrulea Hey, the link for struct Tracked{T} is redirecting to this issue page only. Where is it supposed to link to? Please clarify, in case I am missing something.
@chiragtyagi2003 fixed now
Motivation and description
Currently the
scan
method is used to mark the nodes before applying the actual backpropagation in the graph. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19 We can achieve the same results without employing the this pre-marking of nodes. In this proposal theback
function shall not use thescan
method, but rather take care of the nodes itself, without a pre-marking. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L74The functions and files involved are only where the
ref
field is used: https://github.com/FluxML/Tracker.jl/blob/master/src/Tracker.jl#L41.The following structures/functions/algos have to be understood in order to remove
scan
:back.jl
.Note the function back(g::Grads, x::Tracked, Δ) is used for higher-gradients and we should not focus on this use-case in this proposal.