search

utiasDSL / safe-control-gym

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL
https://www.dynsyslab.org/safe-robot-learning/
MIT License
589 stars 124 forks source link

Example Using PPO (Or any Controller) With Real Crazyfile Drone #163

Open zcase opened 1 week ago

zcase commented 1 week ago

I have been trying to use the Crazyflie 2.x python api to control a drone using the trained PPO controller. I think I might be running into reference frame issue. Do yall have any examples of taking a trained controller like the PPO Stab model and integrating it into a real drone?

Federico-PizarroBejarano commented 6 days ago

Hi @zcase, yes I have used PPO models trained in simulation on real crazyflies before. However, it is never easy unfortunately. How are you controlling the drone? In our gym, the controllers operate by sending direct thrust commands to the four rotors. However, that is not the way most drones have their control interfaces set up. When I do real experiments, I actually switch to using an interface native to the crazyflie, either position, attitude, or rate commands, and train a controller that sends those commands. To do that I use the crazyflie interface created for this paper: https://arxiv.org/pdf/2308.16743. The main branch that does this I believe is https://github.com/utiasDSL/safe-control-gym/tree/beta-iros-competition, but it is unfortunately super out of date. In my own fork i have been doing real crazyflie experiments using https://github.com/Federico-PizarroBejarano/safe-control-gym/tree/new_crazyflie, which works but also has a lot of experiment specific stuff on it.

I am happy to help you get the real crazyflies working, but unfortunately I have found it is a relatively difficult process that is super specific to the drone, task, and controller you are using.

zcase commented 6 days ago

@Federico-PizarroBejarano This is great information. I have been trying to control the drone via the motor commands as outputed from training with this repo and the gym-pybullet-drone repo. I moved to this one because it seemed to have more of the stuff to aid in actual drone flight such as the random ranges in weight and other stuff so that the model doesn't suffer from model mismatch errors when it goes onto the real drone.

So are you saying rather than controlling the drone to go to a point, stabilize, or follow a trajectory using the 4 motor outputs, you train it to use a different interface?

Ill have to checkout out your repo and see what you are doing. I would really like to overcome the sim2real problem with this crazyflie and any input would be helpful! Thank you!

Are there any specific interfaces you use to control the real crazyflie? I have been working with the Crazyflie python API in a script that I built and just been using that but I believe I might be dealing with rotation matrix issues between what this repo trains the ppo algorithm with and what the actual telemetry the drone outputs.

Federico-PizarroBejarano commented 6 days ago

Yeah essentially controlling the real drone using direct motor commands seems to suffer from two main issues:

  1. It is very hard and unstable. Controlling via a higher-level interface is far more stable because it uses the in-built stabilization and tracking controllers of the drone (PID or Mellinger). This should be obvious: controlling the drone using position or velocity setpoints is obviously way easier than controlling it via direct motor commands. Additionally, direct motor commands need to use extremely high rate control, usually 100Hz-500Hz, which is more than I can do for a few reasons. And it needs very accurate state measurements, which also I cannot provide at a sufficiently high accuracy or rate.
  2. The crazyflie has a few control interfaces: https://crazyswarm.readthedocs.io/en/latest/api.html#pycrazyswarm.crazyflie.Crazyflie.cmdFullState. I do not think direct motor control is one of them. Although you can build your own interface, I chose to use their well-supported interfaces. However, this comes with its own issues because then, during training, you need to simulate their whole stack, including their interface, lower-level controllers, etc. That is exactly what the branches I sent do
zcase commented 5 days ago

@Federico-PizarroBejarano I agree that it is a hard problem, but has any one from your team controlled the crazyflie via direct motor control? I was under the impression that was what people were doing in some of the research. Or are they at one step hire like controlling via body axis and thrust command?

I guess what level are yall usually doing your research at?

Federico-PizarroBejarano commented 5 days ago

Hi @zcase, no one in my group has even attempted direct motor control of crazyflies. Part of this is because we work a lot with MPC, and its simply too slow to control a drone via direct motor commands. To my knowledge some of my lab mates in Munich have been able to control the drone in the x-z plane successfully with attitude rate commands, ie roll, pitch, yaw rates + overall thrust (RPYT). I don't think their work has been published yet or anything, but if this is of interest to you I can speak to them. I have gotten it to work in 3D with attitude commands, but not attitude rates.