search

imgeorgiev / ddp

Differential Dynamic Programming (DDP) with automatic symbolic differentiation
MIT License
37 stars 13 forks source link

Can I view the DDP with parameter: "with_hessians=False" as iLQR algorithm? #1

Closed CodingCatMountain closed 5 months ago

CodingCatMountain commented 5 months ago

Hi, @imgeorgiev . Thanks for sharing this repo. And the blog is very wonderful for the greenhand like me to learn DDP algorithm. May I ask a question about DDP and iLQR? I wander about if I set the parameter "with_hessians=False", whether the DDP algorithm can be viewed as iterative LQR algorithm or not? I am looking forward to your reply.

imgeorgiev commented 5 months ago

Yes, DDP without the hessians is iLQR, which works very well and is not nearly as computationally intensive.

CodingCatMountain commented 5 months ago

Thanks! @imgeorgiev May I ask, Do you have any plan to implement guided policy search algorithm which utilizing DDP/iLQR as trajectory generator for policy leanring? Actually, the motivation for me to learn DDP/iLQR is that I want to learn this effective and elegent RL algorithm. :)

imgeorgiev commented 5 months ago

That would be a cool project but it is definitely out of the scope of this repo. What I have implemented here is for easy of understanding rather than efficiency. For a real implementation you probably want to use some library for automatic differentiation (e.g. pytorch)