search

JuliaDiff / DifferentiationInterface.jl

An interface to various automatic differentiation backends in Julia.
https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface
MIT License
192 stars 13 forks source link

Missing `value_and_pushforward_split` #171

Closed prbzrg closed 7 months ago

prbzrg commented 7 months ago

Will there be a value_and_pushforward_split?

gdalle commented 7 months ago

From an autodiff theory perspective it doesn't really make sense, because reverse mode is actually split between forward and reverse sweep, but forward mode only does a forward sweep. In other words, splitting doesn't work or improve performance in forward mode

gdalle commented 7 months ago

You can read https://arxiv.org/pdf/2403.14606.pdf#page145 to see the justification of split reverse mode

gdalle commented 7 months ago

What would indeed speed up forward mode is batched/chunked derivative evaluation, which is something I plan to implement soon

prbzrg commented 7 months ago

Thanks for the answer.

prbzrg commented 7 months ago

I was looking for AbstractDifferentiation.value_and_pushforward_function equivalent that got to this question. It would be nice to have a table about the way to migrate from AbstractDifferentiation to DifferentiationInterface.

prbzrg commented 7 months ago

Although most of the APIs are similar.

gdalle commented 7 months ago

I was looking for AbstractDifferentiation.value_and_pushforward_function equivalent that got to this question.

In this case you can just use the pushforward function when you don't need the value

gdalle commented 7 months ago

It would be nice to have a table about the way to migrate from AbstractDifferentiation to DifferentiationInterface.

Good point, I have opened an issue to keep track: https://github.com/gdalle/DifferentiationInterface.jl/issues/172