search

h2r / sloop

Spatial Language Object-Oriented POMDP (SLOOP)
https://h2r.github.io/sloop
Apache License 2.0
7 stars 2 forks source link

Detail of running on AirSim #4

Open iseecow opened 3 months ago

iseecow commented 3 months ago

I am trying to run AirSim 1.8.1 on Windows, but the drone often collides with buildings. Could you provide more details on running AirSim so that the cause can be found?

zkytony commented 3 months ago

I also observed this with AirSim initially. I tried mitigating by checking for collisions in the transition model and assign large penalty when collision happens in the reward model. It got much better. I was able to consistently get trials without collisions. But since the planner has randomness, it didn't eliminate the occurrance.

Btw all the notes I have about AirSim are here: https://github.com/h2r/sloop/wiki/Running-SLOOP-object-search-on-AirSIm-UE4

iseecow commented 3 months ago

Great! After following the notes in wiki, the drone can sometimes succeed in finding the vehicle! While there still exists some questions: 1. The target car might be showing up in the images captured by the drone's camera, but it has not yet been located. This may due to the fact that the red car was manually placed in the Neighborhood environment. Is it possible for the car to be generated automatically based on specific coordinates?

  1. What`s the effect of parameter "target_poses" in config?
  2. The drone's action is sometimes labeled as "find," yet the car is not visible in the rendered image within the drone's field of view. Could this issue be caused by the misplacement of the target car?
zkytony commented 3 months ago

Answering to the best of my memory since it's been a while:

  1. I'd say of course and it's an Unreal Engine question; For my demos, I just created new scenes with fixed car placements.
  2. That is the groundtruth location of the target object in the 20x20 grid. From the wiki, you will notice that I mapped the neighborhood environment in UE4 to a 20x20 grid. In this demo, in case you didn't notice, there's actually no real object detector being run; the observations were simulated based on the groundtruth location (with some noise) of the target on this grid.
  3. It might be related to my point above. If you encounter this case, feel free to attach a screenshot and I could help you diagnose.