search

microsoft / AirSim-NeurIPS2019-Drone-Racing

Drone Racing @ NeurIPS 2019, built on Microsoft AirSim
https://microsoft.github.io/AirSim-NeurIPS2019-Drone-Racing/
MIT License
356 stars 89 forks source link

Computer Vision Mode #72

Closed rachthree closed 4 years ago

rachthree commented 4 years ago

Hello,

Does Computer Vision mode (like in base AirSim) have full functionality in AirSim-NeurIPS? The reason why we are attempting CV mode is that our perception team members would then have a more flexible way of testing our perception algorithm rather than relying on PX4 (which we could not get to work on our computers), coming up with an unsupported method of keyboard control, or placing objects/drones. After getting a message that the drones were not compatible in CV mode, I had this as my settings.json:

{
  "ClockSpeed": 1,
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "ComputerVision"
}

This worked, except it could only load the box level. It could not load the soccer field, Zhang Jia Jie, or MS building levels. Is there any workaround to this? I also tried loading it with SimMode: Multirotor and kept both drones with one of them having VehicleType: ComputerVision and the other had SimpleFlight but this didn't work out as planned either. I think I saw in #12 that the .pak files cannot be loaded in UE4, so the AirSim.exe provided by you is the only way we can view these levels with gates and drones, correct?

Thank you for any suggestions you may have! AirSim overall seems like a great tool for computer vision, controls, and autonomy work.

saihv commented 4 years ago

Which binary did you use when you tested the computer vision mode?

You can use that same settings.json script you have for Computer Vision mode with the NeurIPS binaries, i.e., the ones with Soccer Field, Zhangjiajie etc. I'd recommend downloading the right pak files and the binary, and then running the binary with the computer vision settings.

rachthree commented 4 years ago

Last message was cut short... edit:

I'm currently using the Windows binaries from https://github.com/microsoft/AirSim-NeurIPS2019-Drone-Racing/releases with airsim_neruips 0.3.0. I downloaded again in case I had an older/wrong version, and it works! However, encountered a small bug but it's not sim-breaking. When going to Manual camera mode and back to FPV, the keyboard controls no longer work properly.

Didn't meant to close this earlier but guess we can consider this closed. Thanks! However, Is there any way to show the drones in CV mode?