Automatically Finding the Values Without Plots?

[chigkim]

Hi, I'm visually impaired, so I can't look at the plots. I'm wondering if it's possible to implement a feature for the script to find the right values, so users don't have to look at the plots and decide. If not, do you have some suggestions on how I can interpret the data without visualization? Thanks,

[titu1994]

Hi @chigkim.

So currently, there isn't any way to automatically determine the cyclic learning rate without a visual check (at least not in literature, and probably not in FastAI library either).

However, looking at it from the point of how we are inspecting a curve, it seems there may be a way to do an approximate finite difference check.

My thought is that we want to observe regions where the loss increases sharply due to high learning rates. Therefore we can compute the approximate finite differences of the plot that is generated. Choosing a single step in the array as the moving window, we can scan the plot left to right to see where the "difference" increases sharply (sharpness is heuristic measure, can be 2x the difference from previous 5 steps). When the process stops, we can pick the learning rate from 5 steps before this explosion and call it a "good" estimate.

Of course, this is really an approximation, and can easily falter. But its the best I can think of at the moment.

[chigkim]

Thanks, I also opened an issue at FastAi, and they're also looking into implementing the feature. Perhaps you could also give some insights as well. https://github.com/fastai/fastai/issues/2072#issuecomment-491554742

[hollowgalaxy]

@titu1994 there is now a suggestion=True feature. See the link above