search

utiasDSL / gym-pybullet-drones

PyBullet Gymnasium environments for single and multi-agent reinforcement learning of quadcopter control
https://utiasDSL.github.io/gym-pybullet-drones/
MIT License
1.19k stars 349 forks source link

How to initialize drone in random positions? #90

Open HimGautam opened 2 years ago

HimGautam commented 2 years ago

I am doing reinforcement learning and for the generalization, I need to initialize drone in random position and orientation. But in this simulator when I try using env = BaseAviary(initial_xyzs=[...]) in each episode, I get an error related to physics plugin. Can you please tell me what can be done?

JacopoPan commented 2 years ago

Hi @HimGautam,

could you please add the terminal printout of the error you encounter here?

HimGautam commented 2 years ago

Hi @JacopoPan , I used env.close() when the episode ended and it solved the problem. I am currently trying to solve the trajectory tracking problem. Can you please give me some insight on how I can approach this problem. I have trained a Neural Network to track an individual point in space but how can I use this to create a trajectory?

JacopoPan commented 2 years ago

Hi @HimGautam

I'd recommend you look at Table 2 and The Supplemental Material of this paper by Yunlong Song. Typical ways include augmenting the observation with target waypoints and dense rewards. Achieving it only from sparse reward is definitely more challenging but an interesting research question, imho.

HimGautam commented 2 years ago

Hi @JacopoPan, Thanks for the suggestion, I am training the agent for tracking the position. I also wanted to ask, is there a quadcopter model available for DroneModel.CF2X. I want to use model predictive control for the tracking problem. My ultimate goal is to use mpc and reinforcement learning for safe learning. So, if possible can you please provide me the dynamics model for that drone. Also,it would really help if there is any implementation of MPC using your simulator.

JacopoPan commented 2 years ago

Hi, @HimGautam,

much of what you are looking for is already in this other project https://github.com/utiasDSL/safe-control-gym (although still only for a quadrotor moving in the x-z plane, for now). Did you have a look at it? There is a supporting paper as well https://arxiv.org/abs/2109.06325

HimGautam commented 2 years ago

I'll take a look at it and get back to you.