search

loco-3d / crocoddyl

Crocoddyl is an optimal control library for robot control under contact sequence. Its solver is based on various efficient Differential Dynamic Programming (DDP)-like algorithms
BSD 3-Clause "New" or "Revised" License
858 stars 174 forks source link

BoxDDP capabilities under v1.2.1 #754

Closed julesser closed 4 years ago

julesser commented 4 years ago

A few weeks ago I created a shooting problem under v1.1.0 based on your bipedal_walking_ubound.py and solved it successfully with the BoxDDP for our robot. After updating to the latest version the solution does not converge anymore after the fourth footstep.

Have there been any changes in the solver structure or similar that could account for this? If not, do you see other possible explanations?

wxmerkt commented 4 years ago

Changes to BoxQP were between v1.0.0 and v1.1.0 - and there are numerical differences in the behaviour (cf. #743). It mostly relates to the old BoxQP working better with BoxDDP, and the new BoxQP working better with BoxFDDP.

The release notes do not indicate related changes between v1.1.0 and v1.2.1 - apart from adding the virtual keywords. There were previously bugs due to the omission of the virtual keyword, so I wonder if that could be related?

Do you have logs of the verbose output from before / after? This can help by looking at the change of costs over iterations.

julesser commented 4 years ago

Thanks for your response. The current log looks like this: logNewBoxDDP.txt. Previously the problem could be solved within ~30-40 iterations. Apparently, I can't reproduce the previous solution with my current setup.

cmastalli commented 4 years ago

Changes to BoxQP were between v1.0.0 and v1.1.0 - and there are numerical differences in the behaviour (cf. #743). It mostly relates to the old BoxQP working better with BoxDDP, and the new BoxQP working better with BoxFDDP.

@wxmerkt -- @julesser reported to develop his problem under v1.1.0, not v1.0.0.

In v1.0.0, we had an experimental code for the BoxQP and the BoxDDP. Both were buggy and fixed in v1.1.0. The latest version of BoxQP uses Cholesky decomposition, we are investigating if this affects the numerical behavior.

Have there been any changes in the solver structure or similar that could account for this? If not, do you see other possible explanations?

I have introduced a quadratic penalization of the friction cone. I suspect this might be the reason of the difference in your behavior. You might want to check the behavior of your application by turning off this cost.

Additionally, could you share with us the log of the solver?

The bipedal.py (or any example related file) might change in future. I suggest you to use them as "example code" for developing your own application.

julesser commented 4 years ago

You might want to check the behavior of your application by turning off this cost.

Thank you for pointing that out. Unfortunately, excluding these costs has not resolved the issue.

Additionally, could you share with us the log of the solver?

Just posted it in the previous comment.

cmastalli commented 4 years ago

It is weird because we have not modified any solver / algorithm between v1.1.0 - v1.2.0. It would be helpful if you could share the exact commit you were using.

I personally checked that the examples have the same behavior. However, we have observed that sometimes happen differences between different PCs. Are you using the same PC?

The current log looks like this: logNewBoxDDP.txt. Previously the problem could be solved within ~30-40 iterations.

The bipedal formulation needs to be improved. When you move from to the next problem, it assumes that the relative CoM position w.r.t. the feet is the same. Therefore, the previous solution needs to reach the final CoM.

It seems for some reason, the final CoM position is not reaching the desired position as before. You could quickly try to do:

  1. increase the stopping threshold, and / or
  2. use the BoxFDDP solver.

Let me know if this fixes your issue.

julesser commented 4 years ago

I used the binary versions, i.e. robotpkg-py36-crocoddyl, both on the same PC.

  1. use the BoxFDDP solver.

Using BoxFDDP works! In contrast to BoxDDP, the solution converges after ~20 iterations: logBoxFDDP.txt

  1. increase the stopping threshold

You mean the maxIter parameter? Increasing it to 1e3 brought no success.

The bipedal formulation needs to be improved. When you move from to the next problem, it assumes that the relative CoM position w.r.t. the feet is the same. Therefore, the previous solution needs to reach the final CoM.

How would your preferred solution look like? Maybe I could contribute to the bipedal formulation under your guidance. We are actually just starting to prepare my master's thesis in this field, where one part is to generate optimal walking trajectories for a new humanoid robot, e.g. with your Crocoddyl library.

cmastalli commented 4 years ago

Using BoxFDDP works! In contrast to BoxDDP, the solution converges after ~20 iterations: logBoxFDDP.txt

Box-FDDP always outperforms Box-DDP. It is a novel method that I have submitted few weeks ago. As soon the paper get accepted, I will share it.

You mean the maxIter parameter? Increasing it to 1e3 brought no success.

No, I meant th_stop. Try something like 1e-9, it seems you're using a value around 1e-6.

cmastalli commented 4 years ago

Maybe I could contribute to the bipedal formulation under your guidance. We are actually just starting to prepare my master's thesis in this field, where one part is to generate optimal walking trajectories for a new humanoid robot, e.g. with your Crocoddyl library.

It would be great! Probably, you should described (in private via email) your project. With this, I will have a better idea how you could contribute.

julesser commented 4 years ago

No, I meant th_stop. Try something like 1e-9, it seems you're using a value around 1e-6.

Thanks for pointing this out. 1e-9 does not seem to be feasible. th_stop = 1e-8 again fails after the second shooting problem:
logBoxDDP_th_stop_e-8.txt.

cmastalli commented 4 years ago

1e-9 does not seem to be feasible. th_stop = 1e-8 again fails after the second shooting problem: logBoxDDP_th_stop_e-8.txt.

Thank you for sharing these results. It would be useful if you can share your script (or give me access to your code).

julesser commented 4 years ago

Thanks for your quick following up. The working version (adapted from bipedal_walk_ubound.py) is located here: https://github.com/julesser/crocoddyl/blob/devel/examples/RH5/walk.py

cmastalli commented 4 years ago

@julesser I reviewed the code twice. It is hard to give you an answer if you do not provide the commits (and replicate the conditions) in which your script was working.

I have strong doubts that the BoxDDP explains your problem. I believe you started to write your script (because it is based on our examples) after the BoxDDP was working (integrated in January 5 and release February 13 (under v1.1.0), 3d3b9470563e6b7c860679d72ee220642658cc68). BoxQP and BoxDDP have not being changed since that release. I just improved the efficiency in c50482471d635f6cf072ab3b3fcfda31e83d8a2b, 4a638a3404b084b2b928831a439af5b1019c0be9, cebc12d3eaa8a2feb7e99b5aaf3240ef233, and fixed an issue regarding infeasible initial state (but it does not seem to be your case) here a504b13424517ea56ed2fb83f5efed. All the tiny changes were tested against each example.

On the other hand, the bipedal formulation has also changed a little in time. Some of the changes are

  1. included the friction cone c1a2a44afaa9ccfe073768483ff02d5c924448bf .
  2. modified the weights of few cost functions 48c4bf441d9ba1c67f12b35f03caa668ebd1b3a6.

Point 2 was done before your modification (before January 5), so it might not be the reason. Instead point 1 was introduced January 2. Both were release in v1.1.0 (February 13). Therefore, this invalidates also the point 1.

The only explanation that I see if you exchanged the biped.py between different versions. It could happen with dirty installation from source. If that happens, then point 1 and 2 could be the explanation.

As far as I am concerned, if you cannot reproduce both cases again then I will have to close this issue (because there is no evidence that the BoxDDP has an issue).

Note that @wxmerkt reported a difference numerical behavior of the BoxQP (used in BoxDDP). I changed completely the BoxQP solver when I integrated in January 5. Previously, BoxQP and BoxDDP were experimental code that there were not working, and at that moment, I did not integrate the bipedal example that you are using. For sure, this is not an explanation of this issue. Indeed, the @wxmerkt's report is not a real bug, it is a need to improve the numerical behavior of this solver.

cmastalli commented 4 years ago

I will close this issue since there is no activity or evidence to have an issue in the BoxDDP.

Feel free to re-open it if things change.