Flying across the gap task failure.

[jc-bao]

❓ Issue

1. The drone hit the wall.
2. Cannot stabilize the object while only training stabilization task is successful.

video demo

🤔 Possible solutions

1. Hit wall ▶️ Reward engineering
2. Stabilize object ▶️ Curriculum.

[jc-bao]

🔧 Trail

🧰 Reward Engineering

• Move faster: remove the velocity penalty.
• Do not hit the wall: increase the hit penalty.

👀 Results	P_hit x 3	P_hit x 3 + P_v x 0	P_hit x 3 + P_v x close to target
😢 not move cross the gap	😢 unable to stabilize the object

😢 Not working

[jc-bao]

🔧 Trail

🧰 auto reset + Curriculum of gap size

👀 Result

😄 Hitting wall problem is resolved. 😢 Still cannot stabilize the object.

🏃🏻 Next step

Vanalli stabilization task is successful while the combined task is not working ▶️ Try stabilize to a certain point without wall blocking.

[jc-bao]

🔧 Trail

🧰 only stabilize without the gap

👀 Results

13cm error + 0.6m/s velocity error

🧰 only stabilize velocity

👀 Results

0.4m/s velocity error

[jc-bao]

🤗 Summary

• use auto reset to stop the hit exploration.
• use gap size curriculum to make the policy more stable.