Closed rejuvyesh closed 2 years ago
I tried adding back an approximate version of onecycle
based on existing schedulers in ParameterSchedulers
but it's not exactly the same.
I guess you want the ability to have a sequence of two sine waves with different periods, and the second starting at an offset t == half period of first sine
?
With ParameterScheduler@0.3.1, you should be able to do this like:
function onecycle(
nsteps, max_val;
pct_start=0.25,
div=25, divfinal=1e5,
start_val=max_val / div, end_val=max_val / divfinal)
warmup = ceil(Int, nsteps * pct_start)
warmdown = nsteps - warmup
Sequence(Sin(λ0=max_val, λ1=start_val, period=2*warmup) => warmup,
Shifted(Sin(λ0=max_val, λ1=end_val, period=2*warmdown), warmdown + 1) => warmdown)
end
I think the docstring should be updated to:
Callback for hyperparameter scheduling. Takes pairs of [`HyperParameter`](#)
types and [ParameterSchedulers.jl schedules](https://darsnack.github.io/ParameterSchedulers.jl/dev/README.html).
Fixes #106.
Not sure if you want to keep an equivalent of
onecycle
.