The drone hovering at a fixed point close to the origin instead of converging to the center.

[jc-bao]

Test case: fixed mass to 0.01, fixed the constant disturbance force to 0.8 times object mass*gravity, fixed damping parameter to 0.

[jc-bao]

Check the policy at the origin point

Make sure the drone is initialized at the centre, then it can hover here.

• actor output at the origin: tensor([-0.5308, 0.7817])
• expected torque: 0.80.019.8 = 0.784 ✅
• expected thrust: $\sqrt{(0.784+0.98)^2+0.784^2}=1.930$❌
• Jacobian at the centre:
  • Regarding the position: [ 0.0605, -0.9488, 0.3043] -> Not stable for x and theta
  • Regarding the velocity: [0.2593, -0.7368, -0.3238] -> Not stable for $v_x$. (Which aligns with our observation. )

[jc-bao]

Check the policy without the fixed disturbance

Final error is large	Cannot fully converge to zero along y	speed quickly converge to zero during the training

Meanwhile, the PID controller can steer the agent to zero quite precisely.

Conclusion: RL-related issue.

[jc-bao]

Update RL to use a non-linear reward. ✅

Current reward function regarding the distance to the centre:

• Final error: 0.006m
• Policy:

[jc-bao]

After adding all uncertain parameters

• Final distance to center = 0.027
• Learned policy

pi	vis	plot
Experterror=0.027
Vanilla error=0.106