search

martinResearch / MatlabAutoDiff

A matlab implementation of forward automatic differentiation with operator overloading and sparse jacobians
BSD 3-Clause "New" or "Revised" License
39 stars 11 forks source link

Bug in AutoDiff? #19

Open bharswami opened 3 weeks ago

bharswami commented 3 weeks ago

Withdraw the issue. It seems fixed.

bharswami commented 3 weeks ago

I would however like to know this: Do we have to call AutoDiffJacobianAutoDiff every time to get the Jacobian at different points? Is the auto differentiated code available so that that can be called instead?

martinResearch commented 3 weeks ago

Yes we have to call AutoDiffJacobianAutoDiff every time to get the Jacobian at different points. There is no differentiated code generated that could be reuse. This library implements forward autodiff through operator overloading. You would need to use a library that does autodiff through source code transformation if you want to have differentiated code (see implementation section here https://en.wikipedia.org/wiki/Automatic_differentiation).

bharswami commented 3 weeks ago

Can you suggest a similar AutoDiff package for C++? Something that calculates Jacobian.

bharswami commented 3 weeks ago

I am looking to save on computation time. Or is there a way to make AutoDiff in Matlab faster?

bharswami commented 3 weeks ago

Also, when calling the function to be differentiated, do we need to set the intermediary variables to type AutoDiff? If I don't, I get the error, cannot convert from type "AutoDiff" to "double".

martinResearch commented 3 weeks ago

"Can you suggest a similar AutoDiff package for C++? Something that calculates Jacobian." There a many C++ packages to do autodiff with various pro and cons. I have not experience with autodiff in C++ a quick seqrch on google lead to https://github.com/autodiff/autodiff, mayeb worth lookin into?

"is there a way to make AutoDiff in Matlab faster?". Not sure, you can give a try to atlernative libraries mentioned in the readme. As said in the readme It is likely that the speed could be improved by representing Jacobian matrices by their transpose, due to the way Matlab represents internally sparse matrices. That would necessitate quite a bit of work. It is also sometimes possible to get a significative speed up by rewriting diffrently the function that you want to differentiate (see https://github.com/martinResearch/MatlabAutoDiff/issues/1#issuecomment-812699747).

"Also, when calling the function to be differentiated, do we need to set the intermediary variables to type AutoDiff?" that should be done automatically. Have a look at the examples to see that. Do you have a small example you can share to reproduce the problem you are getting?

bharswami commented 3 weeks ago

"Can you suggest a similar AutoDiff package for C++? Something that calculates Jacobian." There a many C++ packages to do autodiff with various pro and cons. I have not experience with autodiff in C++ a quick seqrch on google lead to https://github.com/autodiff/autodiff, mayeb worth lookin into? I checked this. It is way too slow for my liking. "is there a way to make AutoDiff in Matlab faster?". Not sure, you can give a try to atlernative libraries mentioned in the readme. As said in the readme It is likely that the speed could be improved by representing Jacobian matrices by their transpose, due to the way Matlab represents internally sparse matrices. That would necessitate quite a bit of work. It is also sometimes possible to get a significative speed up by rewriting diffrently the function that you want to differentiate (see #1 (comment)). I modified the AutoDiff code to treat derivatives as "sparse". Also I am using the Jacobians as sparse, since the optimization toolbox I use needs it so. I will have to see if I can write my function differently. "Also, when calling the function to be differentiated, do we need to set the intermediary variables to type AutoDiff?" that should be done automatically. Have a look at the examples to see that. Do you have a small example you can share to reproduce the problem you are getting? This doesn't happen automatically when I use it. Suppose a is initialised as double(zeros(10,1)), b as AutoDiff(zeros(10,1)). "a=a + b" doesn't work.

martinResearch commented 1 week ago

In the tests I have the lines

a = randn(3, 3); f = @(x) a + x ; CheckAutoDiffJacobian(f, randn(3, 3), 1e-7);

The array a is a normal matlab array and it works. I don't have matlab installed to test your line of code but it seems to do the same operation as the lines above.