Error on computing gradients when `training isa Val{false}`

LuxDL / LuxLib.jl

Backend for Lux.jl

MIT License

7 stars 0 forks source link

Error on computing gradients when `training isa Val{false}` #98

Open avik-pal opened 1 month ago

avik-pal commented 1 month ago

We need to do this for:

[x] ChainRules
[ ] Enzyme
[x] ReverseDiff
[x] Tracker
[x] ForwardDiff

A possible implementation would be to define assert_training_in_rrule

When called outside of ChainRules / EnzymeRules this is a no-op.
Otherwise we throw an assertion error

avik-pal commented 1 month ago

This can also serve as a prototype for an auto-training mode using determine_istraining function that changes behavior based on whether it was called inside an rrule function