search

utiasDSL / safe-control-gym

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL
https://www.dynsyslab.org/safe-robot-learning/
MIT License
560 stars 123 forks source link

Reproducing CBF approaches in cartpole #155

Open luigiberducci opened 1 month ago

luigiberducci commented 1 month ago

Hi everyone, I wanted to reproduce the experiments with CBF, CBF-NN and RL Safe Explorer in the cartpole environment, but I am struggling to make sense of the results of the examples. Running the cbf example with ´safe-control-gym/examples/cbf/cbf_experiment.sh´ I obtain the following pictures:

Figure_2 Figure_3

They suggest there are action corrections but the simulations show the pole repeatedly falling down. Is it the expected behaviour?

Thanks a lot for your help. I look forward to hearing from you.

Best, Luigi

Federico-PizarroBejarano commented 1 month ago

Hello Luigi, I am very sorry for the delay. I saw the same results when I ran it, which is unexpected. After some troubleshooting, I realized some recent PRs broke the CBF experiment scripts, which I have now remedied here: https://github.com/utiasDSL/safe-control-gym/pull/156. Feel free to pull and it should now work again, or copy the fixes in the relevant files from that PR. Sorry for the problem and thank you for catching it!!

Now more generally, we have not developed our CBF, CBF_NN, or Safe-Explorer code very much. They work for the basic examples provided, but I am not confident in their extendability to more complex scenarios. if you manage to improve them, we would be very happy if you contributed any code!! Let me know if I can help any more, thanks again!

luigiberducci commented 1 month ago

Thanks Federico! Now it looks much better. About the experiments, I see in the that the cbf and cbf-nn are implemented for the cartpole, and for the safe-explorer for all the tasks. Could you please confirm I am not missing any other relevant file? :pray:

I will be happy to share my implementation once finished!

Federico-PizarroBejarano commented 1 month ago

Hi Luigi, that's great I am happy it is working! And yes that is correct, the CBFs were never extended to the quadrotor systems unfortunately. If you manage to get it working well for them that would be great! Thanks for your interest in the gym and your work with it!