Currently, we already have the Checkpointer struct, which saves the model every epoch.
However, this can blow up in file size, and usually we are only interested in the best, and maybe last checkpoints.
Possible Implementation
From epoch two, we have to consider the (logical) checkpoints (prev, current, best) .
If prev != best, delete prev.
If metric(current) > metric(best), delete path[best].
We could implement this by making the Checkpointer callback mutable with a Dict Dict{Symbol, String}(:prev=>"", :current=>"", :best=>"").
Then, implementing the above logic would roughly be
function update!(cb::Checkpointer, new_path, new_metric; ord::Base.Ordering=Base.Order.Forward)
(cb.models[:latest] != cb.paths[:best]) && rm(cb.paths[:latest])
cb.models[:latest] = (;path=>new_path, metric=>new_metric)
if lt(ord, cb.models[:best].metric, cb.models[:latest].metric))
rm(cb.models[:best].path)
cb.models[:best] = cb.models[:latest]
end
end
Motivation and description
Currently, we already have the
Checkpointer
struct, which saves the model every epoch. However, this can blow up in file size, and usually we are only interested in the best, and maybe last checkpoints.Possible Implementation
From epoch two, we have to consider the (logical) checkpoints
(prev, current, best)
. Ifprev != best
, deleteprev
. Ifmetric(current) > metric(best)
, deletepath[best]
.We could implement this by making the Checkpointer callback mutable with a Dict
Dict{Symbol, String}(:prev=>"", :current=>"", :best=>"")
. Then, implementing the above logic would roughly be