search

isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
2.3k stars 947 forks source link

[Question] QuadCopter training with camera #1445

Closed JulienHansen closed 3 days ago

JulienHansen commented 4 days ago

Hi,

Apologies if this question has been asked before or if the answer is obvious. I’m still relatively new to reinforcement learning, robotics, and simulation, having started just a month ago.

I’m currently working on a reinforcement learning task for a drone that needs to navigate through a level to reach a desired position on the opposite side. To achieve this, I’ve added a camera to the drone for perception.

My question is: how should I approach training for such an application? In your opinion, what would be the fastest and most effective way to implement this?

Would it be better to:

  1. Build my own library and custom wrappers?
  2. Modify an existing library like RSL-RL?
  3. Consider another approach entirely?

Thanks in advance for your responses

shaoxiang commented 3 days ago

I also really want to know that currently the quadcopter example only has a direct version, not a manage_based version, and I don't know if it's possible to implement a manage_based version because it doesn't seem to be possible to implement it through ActionsCfg's EffortActionCfg, etc. Therefore, I personally think that it may be difficult to implement a ready-made visual quadcopter.

RandomOakForest commented 3 days ago

This is a great discussion. Will move it into our Discussions section for the team to follow upon. Thanks for posting this.