search

rfeinman / pytorch-minimize

Newton and Quasi-Newton optimization with PyTorch
https://pytorch-minimize.readthedocs.io
MIT License
292 stars 34 forks source link

feature request minimize_scalar #5

Closed 7starsea closed 2 years ago

7starsea commented 2 years ago

I would like to see a similar torch minimize_scalr api corresponding to scipy.optimize.minimize_scalar.

rfeinman commented 2 years ago

Hi @7starsea,

Thanks for the suggestion - I had a quick look at the scipy implementation. Although I may include minimize_scalar at some point in the future, it's not a high priority right now.

The distinguishing features of pytorch-minimize vs. scipy are: 1) automatic differentiation, and 2) GPU acceleration. The algorithms for minimize_scalar don't use gradient information, therefore autograd is unnecessary. Furthermore, one-dimensional (scalar) optimization problems generally don't solicit GPU acceleration. Therefore, I see little advantage to a pytorch implementation of minimize_scalar over the existing scipy tool.

I would suggest using scipy for your problem and simply converting between pytorch/numpy in your objective function. The algorithm performance should be roughly equivalent to a native pytorch implementation.

7starsea commented 2 years ago

Thanks for your detailed response. I agree with your point about the distinguishing features of pytorch-minimize vs. scipy.