search

davidtvs / pytorch-lr-finder

A learning rate range test implementation in PyTorch
MIT License
912 stars 116 forks source link

Suggested LR not returned when min_grad_idx is 0 in plot() #65

Closed manuel-munoz-aguirre closed 3 years ago

manuel-munoz-aguirre commented 3 years ago

When using the plot function in a situation like the following: starting_lr_fulltrain_DEBUG where the first value in lrs[min_grad_idx] is the suggested learning rate, even if the suggested learning rate is printed, it is not returned.

Expected behavior: return ax, lrs[min_grad_idx] Observed behavior: return ax

These seem to be the relevant lines. Seems that ax is returned due to min_grad_idx evaluating to False (because it is 0): https://github.com/davidtvs/pytorch-lr-finder/blob/9cfcbecba35866711647a251b4527c6f29b9c9f5/torch_lr_finder/lr_finder.py#L535-L538

NaleRaphael commented 3 years ago

@manuel-munoz-aguirre You are right, the condition should be ... and min_grad_idx is not None: which is the same as line 510 since it's a numerical value: https://github.com/davidtvs/pytorch-lr-finder/blob/9cfcbecba35866711647a251b4527c6f29b9c9f5/torch_lr_finder/lr_finder.py#L508-L512

Thanks for the feedback! I'll send a patch for it.

davidtvs commented 3 years ago

Thanks for raising the issue @manuel-munoz-aguirre and thanks to @NaleRaphael for fixing it in #66

The fix has been merged, closing this issue.