Policy dose not converge in quadrotor transportation scenario.

[jc-bao]

The newly designed quadrotor transportation case cannot hover at a fixed point.

Learned policy

The policy tends to hit the boundary of flying space.

Learning curve

It cannot converge after a long time of training.

Check with PID

The PID controller can track the center point very precisely.

[jc-bao]

Check: can the drone hover at a fixed point when the object weight is set to zero?

• Objective: check if the attitude + thrust control is working.

• Learning Curve: compared with thrust+torque control, the policy does not converge at all.

  

• Learned strategy: also tends to hit the boundary of the flying space.

  

• Check random explore policy: the drone tends to hit the upper boundary & the object moves really quickly. ▶️ Make the thrust smaller + remove object observation. 😢 The result is the same, not the issue.

  

[jc-bao]

Resolved: the dynamic error

Error dynamics cause the problem in the multi-drone case. Be careful with a GPU-based environment during the tensor operation!