disungatullina
commented
3 months ago Hi @TwiceMao,
Thank you for your interest in our work. The method is general and can be applied to any task where the following conditions are met:
- MinBackProp IFT: If you can write down the system of equations whose solution is the output for the forward pass, and the Jacobian is invertible.
- MinBackProp DDN: If you can formulate the optimization problem whose solution is the output for the forward pass, and the function to optimize, along with its constraints, are second-order differentiable.
The Implicit function theorem is widely used in robotics.
The method can shorten time, especially for high computation tasks. The main bottleneck is that a researcher/engineer should write down the system of equations, calculate the Jacobian, and code it by himself.
Hi @disungatullina,
Thank you for your excellent work. I noticed that your proposed algorithm is applied to a specific task in your paper. I am curious if your algorithm could be utilized for general tasks that require backpropagation. Then deep learning will shorten much time.