search

hahnec / torchimize

numerical optimization in pytorch
https://hahnec.github.io/torchimize/
GNU General Public License v3.0
134 stars 6 forks source link

Combined function/jacobian interface #5

Open pkienzle opened 1 year ago

pkienzle commented 1 year ago

With chain rule one often sees many common sub-expressions in the calculation of the function and its derivatives. It would save time to compute the two of these together rather than having one call to the function and one call to the jacobian.

hahnec commented 1 year ago

I see your point to gain computational performance here. However, this would require the user to provide only part of the derivative function? Such an interface would be quite custom/unconventional or at least not straightforward... Feel free to draft this as a separate and more performant function. Again, I am happy to accept PRs.

pkienzle commented 1 year ago

Figuring out a nice interface will take some effort.

In particular I would like it to be able to use the autodifferentiation tools in torch if the function supports them. I also have functions implemented in pure cuda which I'm planning to tie in through cupy. These will need either numerical differentiation or hard-coded analytic forms, so I don't want to require AD.