FPV camera can not send flying drone images back (only the nonFLY drone camera work)

[hbzhang]

Two drones are in racing. I setup camera images like the following. But only the non fly drone send back FPV images. i want to visualize the flying drone real time tracjetory

I use the following callback function:

def image_callback(self):
    # get uncompressed fpv cam image
    request = [asim.ImageRequest("fpv_cam", asim.ImageType.Scene, False, False)]
    response = self.airsim_client_images.simGetImages(request)
    img_rgb_1d = np.fromstring(response[0].image_data_uint8, dtype=np.uint8)
    img_rgb = img_rgb_1d.reshape(response[0].height, response[0].width, 3)
    cv2.imshow("img_rgb", img_rgb)
    cv2.waitKey(1)

[madratman]

You'll have to add a camera to drone_2 in setting.json:

{
  "ClockSpeed": 1,
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor",
  "Vehicles": {
    "drone_1": {
      "Cameras": {
        "fpv_cam": {
          "CaptureSettings": [
            {
              "FOV_Degrees": 90,
              "Height": 240,
              "ImageType": 0,
              "Width": 320
            }
          ],
          "Pitch": 0.0,
          "Roll": 0.0,
          "X": 0.25,
          "Y": 0.0,
          "Yaw": 0.0,
          "Z": 0.0
        }
      },
      "Pitch": 0.0,
      "Roll": 0.0,
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 0.0,
      "Yaw": 0.0,
      "Z": 0.0
    },
    "drone_2": {
      "Cameras": {
        "fpv_cam_2": {
          "CaptureSettings": [
            {
            "FOV_Degrees": 90,
            "Height": 240,
            "ImageType": 0,
            "Width": 320
            }
          ],
          "Pitch": 0.0,
          "Roll": 0.0,
          "X": 0.25,
          "Y": 0.0,
          "Yaw": 0.0,
          "Z": 0.0
        }
      },
      "Pitch": 0.0,
      "Roll": 0.0,
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 0.0,
      "Yaw": 0.0,
      "Z": 0
    }
  }
}

Note that airsim doesn't allow for same camera names, hence, it's called fpv_cam_2.

then you add it to the request list: request = [airsim.ImageRequest("fpv_cam", asim.ImageType.Scene, False, False), airsim.ImageRequest("fpv_cam_2", asim.ImageType.Scene, False, False)]

then you can get the image from fpv_cam_2 from second object in response list (response[1])

[hbzhang]

Thanks for the suggestion.

I changed the setting.json to this: { "ClockSpeed": 1, "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md", "SettingsVersion": 1.2, "SimMode": "Multirotor", "Vehicles": { "drone_1": { "Cameras": { "fpv_cam": { "CaptureSettings": [ { "FOV_Degrees": 90, "Height": 240, "ImageType": 0, "Width": 320 } ], "Pitch": 0.0, "Roll": 0.0, "X": 0.25, "Y": 0.0, "Yaw": 0.0, "Z": 0.0 } }, "Pitch": 0.0, "Roll": 0.0, "VehicleType": "SimpleFlight", "X": 0.0, "Y": 0.0, "Yaw": 0.0, "Z": 0.0 }, "drone_2": { "Cameras": { "fpv_cam_2": { "CaptureSettings": [ { "FOV_Degrees": 90, "Height": 240, "ImageType": 0, "Width": 320 } ], "Pitch": 0.0, "Roll": 0.0, "X": 0.25, "Y": 0.0, "Yaw": 0.0, "Z": 0.0 } }, "Pitch": 0.0, "Roll": 0.0, "VehicleType": "SimpleFlight", "X": 0.0, "Y": 0.0, "Yaw": 0.0, "Z": 0.0 } } }

and image call back like this:

def image_callback(self):

get uncompressed fpv cam image

    request = [asim.ImageRequest("fpv_cam_2", asim.ImageType.Scene, False, False)]
    response = self.airsim_client_images.simGetImages(request)
    img_rgb_1d = np.fromstring(response[0].image_data_uint8, dtype=np.uint8)
    print(f"image heigh is {response[0].height}")
    img_rgb = img_rgb_1d.reshape(response[0].height, response[0].width, 3)
    cv2.imshow("img_rgb", img_rgb)
    cv2.waitKey(1)

But looks like the response = self.airsim_client_images.simGetImages(request) hangs and the CV2 never able to render the response as shown below: