Safe Deep Policy Adaptation. (arXiv:2310.08602v2 [cs.RO] UPDATED) https://ift.tt/gpDI3Wq
A critical goal of autonomy and artificial intelligence is enabling
autonomous robots to rapidly adapt in dynamic and uncertain environments.
Classic adaptive control and safe control provide stability and safety
guarantees but are limited to specific system classes. In contrast, policy
adaptation based on reinforcement learning (RL) offers versatility and
generalizability but presents safety and robustness challenges. We propose
SafeDPA, a novel RL and control framework that simultaneously tackles the
problems of policy adaptation and safe reinforcement learning. SafeDPA jointly
learns adaptive policy and dynamics models in simulation, predicts environment
configurations, and fine-tunes dynamics models with few-shot real-world data. A
safety filter based on the Control Barrier Function (CBF) on top of the RL
policy is introduced to ensure safety during real-world deployment. We provide
theoretical safety guarantees of SafeDPA and show the robustness of SafeDPA
against learning errors and extra perturbations. Comprehensive experiments on
(1) classic control problems (Inverted Pendulum), (2) simulation benchmarks
(Safety Gym), and (3) a real-world agile robotics platform (RC Car) demonstrate
great superiority of SafeDPA in both safety and task performance, over
state-of-the-art baselines. Particularly, SafeDPA demonstrates notable
generalizability, achieving a 300% increase in safety rate compared to the
baselines, under unseen disturbances in real-world experiments.
Safe Deep Policy Adaptation. (arXiv:2310.08602v2 [cs.RO] UPDATED)
https://ift.tt/gpDI3Wq
A critical goal of autonomy and artificial intelligence is enabling autonomous robots to rapidly adapt in dynamic and uncertain environments. Classic adaptive control and safe control provide stability and safety guarantees but are limited to specific system classes. In contrast, policy adaptation based on reinforcement learning (RL) offers versatility and generalizability but presents safety and robustness challenges. We propose SafeDPA, a novel RL and control framework that simultaneously tackles the problems of policy adaptation and safe reinforcement learning. SafeDPA jointly learns adaptive policy and dynamics models in simulation, predicts environment configurations, and fine-tunes dynamics models with few-shot real-world data. A safety filter based on the Control Barrier Function (CBF) on top of the RL policy is introduced to ensure safety during real-world deployment. We provide theoretical safety guarantees of SafeDPA and show the robustness of SafeDPA against learning errors and extra perturbations. Comprehensive experiments on (1) classic control problems (Inverted Pendulum), (2) simulation benchmarks (Safety Gym), and (3) a real-world agile robotics platform (RC Car) demonstrate great superiority of SafeDPA in both safety and task performance, over state-of-the-art baselines. Particularly, SafeDPA demonstrates notable generalizability, achieving a 300% increase in safety rate compared to the baselines, under unseen disturbances in real-world experiments.
via cs.RO updates on arXiv.org http://arxiv.org/