FluxML / Tracker.jl

Flux's ex AD
Other
51 stars 37 forks source link

[gsoc] Remove the `scan` method. #164

Open MariusDrulea opened 6 months ago

MariusDrulea commented 6 months ago

Motivation and description

Currently the scan method is used to mark the nodes before applying the actual backpropagation in the graph. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19 We can achieve the same results without employing the this pre-marking of nodes. In this proposal the back function shall not use the scan method, but rather take care of the nodes itself, without a pre-marking. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L74

The functions and files involved are only where the ref field is used: https://github.com/FluxML/Tracker.jl/blob/master/src/Tracker.jl#L41.

The following structures/functions/algos have to be understood in order to remove scan:

  1. struct Tracked{T} and it's members
  2. the scan, back and back methods in the file back.jl.

Note the function back(g::Grads, x::Tracked, Δ) is used for higher-gradients and we should not focus on this use-case in this proposal.

chiragtyagi2003 commented 6 months ago

@MariusDrulea Hey, the link for struct Tracked{T} is redirecting to this issue page only. Where is it supposed to link to? Please clarify, in case I am missing something.

MariusDrulea commented 6 months ago

@chiragtyagi2003 fixed now