Implementing a Custom Controller on Safe-Control-Gym / Gym-Pybullet Drones

[lakshmikanth-nageswar]

Hi, could anyone please clear these queries of mine:

• Where exactly in the code is the parameters of System Dynamics (Mass, Inertia, etc) is declared (numerical values)? and is the Dynamics used in training the same as PyBullet Crazyflie (Drone) Dynamics? And could you please tell how to discretise the dynamics (for complete non linear model and linearised dynamics about an equilibrium point)

• How to modify the training tasks say for hover, single set-point (regularisation) and tracking problem through this framework? and for visualisation of trained policy in pybullet as well. (how to modify the BaseAviary or VelocityAviary for tracking and other scripts required to be modified?)

• How to implement our own controller (say linear or non-linear or learning based) in this environment

Thanks in Advance.

[Federico-PizarroBejarano]

Hello @lakshmikanth-nageswar, thank you for your interest in our gym. I will answer your points briefly below. Let me know if you need more details.

• System Parameters: The inertial properties can be set in the configuration YAML of the environment, such as here: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/examples/rl/config_overrides/quadrotor_3D/quadrotor_3D_track.yaml#L86. You can additionally set randomizations of these parameters here: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/examples/no_controller/verbose_api.yaml#L36. You can find more information about the inertial properties by searching these parameters in the environments, such as in the cartpole.py or quadrotor.py files.
• Dynamics vs PyBullet: The dynamics provided for each environment are very close but not exact. The actual environment is governed by the PyBullet physics engine, and cannot be represented perfectly by an equation. The difference between the provided equation and the actual evolution of the simulated system is low.
• Discretizing Dynamics: You can find an example of discretizing and linearizing the model here: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/safe_control_gym/controllers/mpc/linear_mpc.py#L84 and simply discretizing it without linearizing here: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/safe_control_gym/controllers/mpc/mpc.py#L138
• Setting Stabilization or Tracking Task: You can choose two general tasks as of now: stabilization to a set-point, or tracking a trajectory. This is also set in the environment YAML of the experiment. Here is a stabilization example: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/examples/rl/config_overrides/quadrotor_3D/quadrotor_3D_stab.yaml#L76. Here is a tracking example: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/examples/rl/config_overrides/quadrotor_3D/quadrotor_3D_track.yaml#L76. Here is a custom trajectory: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/examples/pid/pid_experiment.py#L50
• Visualization: To turn on visualization, simply set the flag in the environment YAML: https://github.com/utiasDSL/safe-control-gym/blob/0cca9d84821e9c29274d1683bfc0e6f650f749de/examples/no_controller/verbose_api.yaml#L6
• Implementing Your Own Controller: To implement your own controller, follow the example of a simple controller in the gym such as https://github.com/utiasDSL/safe-control-gym/tree/main/safe_control_gym/controllers/lqr. You will need to create your own folder in the 'safe_control_gym/controllers' folder, put in a Python file for your controller class, a default YAML config, and add your controller here: https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/__init__.py

[lakshmikanth-nageswar]

Hi, Thanks a lot for your response! It is a lot clearer now to implement.

I am excited to implement a Model-based controller that is capable of learning. Could you please tell me on code level how to implement my controller - structure of codebase for model-based controller aided by learning [Say training the controller offline with the params of PyBullet drone and later using it for visualization (or) training it with pose and other state feedback from Pybullet sim]?

I am excited to hear from you.

[Federico-PizarroBejarano]

No worries! I am not sure exactly how to help you with your current project. What I would do is take a look at our examples folder, especially our RL and MPC examples. If you wanted to make your own controller and own experiments, I would copy an existing controller from the controllers folder that is similar to what you are trying to do (for example, PPO for RL or MPC for model-based), modify it, then copy the experiments you want to do from the examples folder and modify them to run your controller. You can see how we train an RL controller in the examples/rl folder. You can visualize your experiments by passing gui: True in the YAML file of your experiments.

[lakshmikanth-nageswar]

Thanks a lot for clarification. I'll approach you further if facing any issues. Closing the issue for now.