feldberlin / wavenet

An unconditioned Wavenet implementation with fast generation.
3 stars 0 forks source link

Learning rate schedule #23

Open purzelrakete opened 3 years ago

purzelrakete commented 3 years ago

What

Try Adam with LR 2e-4 with two 10x decreases ~80% and ~95% of the way through.

Hypothesis

People seem to do this. Maybe one-cycle isn't the bible after all?

Results

Write up the results of your experiment once it has completed and has been analysed. Include links to the treatment run, and also to the baseline if appropriate.

Acceptance Criteria