search

resibots / blackdrops

Code for the Black-DROPS algorithm: "Black-Box Data-efficient Policy Search for Robotics", IROS 2017/ICRA 2018
Other
64 stars 22 forks source link

How should I include constraints on the state variables #12

Open lydiarong opened 5 years ago

lydiarong commented 5 years ago

In the cartpole scenario,there are no constraints in the state space,but I want to control the position of the cart between ±0.2m,I manually add the constraints in predict_policy function of system.hpp as shown below ,it works well when the constraints is ±2.5m,but when it comes to ±0.2m,it fails. image

costashatz commented 5 years ago

In the cartpole scenario,there are no constraints in the state space,but I want to control the position of the cart between ±0.2m,I manually add the constraints in predict_policy function of system.hpp as shown below ,it works well when the constraints is ±2.5m,but when it comes to ±0.2m,it fails.

This is not the proper way of inserting the constraints.. Any system behavior should be described in the system function. As cartpole is modeled as an ODE, this means that you need to alter the dynamics function.

Nevertheless, I have a feeling that 0.2m is too small and thus there is no space left for the cart to make the movements that would allow the pole to swing-up.

lydiarong commented 5 years ago

Thanks for your reply,I'll try this.But if I want to apply this algorithm to a real physical model,then it's useless to alter the dynamics function even it can work in simulation,right? How should I add the state constraints when it comes to a real physical cartpole model?

costashatz commented 5 years ago

But if I want to apply this algorithm to a real physical model,then it's useless to alter the dynamics function even it can work in simulation,right? How should I add the state constraints when it comes to a real physical cartpole model?

In a physical cartpole, the limits will be on the hardware system, right? Or in the controller implemented on the physical system, no? I mean there will be a physical stop or something in the controller that stops the cartpole (and this should not be given to the algorithm, except if you want to give some prior knowledge)..