Polling Images using C++ API is alot slower than python

[MoBaT]

Bug report

• AirSim Version/#commit: v1.3.1
• UE/Unity version: 4.25 + 4.18
• OS Version: Ubuntu 18.04

What's the issue you encountered?

Whenever I try to run the example C++ code using AirLib compiled with either GCC or clang, I get an FPS of around 5-8. When I run the python equivalent using pip install airsim or running the setup.py build && setup.py install, I get a FPS of 25+. I have tried compiling the C++ for AirLib + RpcLib in GCC/Clang Release (static and shared) and applied compiler optimizations but it stays the same.

Settings

  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor",
  "EngineSound": false,
  "SpeedUnitFactor": 1,
  "SpeedUnitLabel": "m/s",
  "SubWindows":
  [
    {"WindowID": 0, "CameraName": "D435", "ImageType": 0, "Visible": true},
    {"WindowID": 2, "CameraName": "D435", "ImageType": 4, "Visible": true}
  ],
  "TimeOfDay":
  {
    "Enabled": false,
    "StartDateTime": "",
    "CelestialClockSpeed": 1,
    "StartDateTimeDst": false,
    "UpdateIntervalSecs": 60
  },
  "Vehicles":
  {
    "0":
    {
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 0.0,
      "Z": 0.0,
      "RC":
      {
        "RemoteControlID": 0,
        "AllowAPIWhenDisconnected": false,
        "AllowAPIAlways": false
      },
      "Sensors":
      {
        "Imu": {
          "SensorType": 2,
          "Enabled": true
        },
        "Gps": {
          "SensorType": 3,
          "Enabled": true
        },
      },
      "Cameras":
      {
        "CAM1":
        {
          "CaptureSettings":
          [
            {
              "ImageType": 0,
              "Width": 640,
              "Height": 480,
              "FOV_Degrees": 87,
              "MotionBlurAmount": 0
            },
            {
              "ImageType": 1,
              "Width": 640,
              "Height": 480,
              "FOV_Degrees": 87,
              "MotionBlurAmount": 0
            }
          ],
          "X": 0.5,
          "Y": 0,
          "Z": -0.15,
          "Roll": 0,
          "Pitch": 0,
          "Yaw": 0
        }
      }
    }
  }
}

How can the issue be reproduced?

1. Run C++ code and monitor output and see that it is 5-8 FPS.

   
   #include "vehicles/multirotor/api/MultirotorRpcLibClient.hpp"
   #include "common/common_utils/FileSystem.hpp"
   #include <iostream>
   #include <chrono>

double clockToMilliseconds(clock_t ticks){ // units/(units/time) => time (seconds) 1000 = milliseconds return (ticks/(double)CLOCKS_PER_SEC)1000.0; } //...

clock_t deltaTime = 0; unsigned int frames = 0; double frameRate = 30; double averageFrameTimeMilliseconds = 33.333;

int main() { using namespace msr::airlib;

auto client = new msr::airlib::RpcLibClientBase("localhost", RpcLibPort);
typedef ImageCaptureBase::ImageRequest ImageRequest;
typedef ImageCaptureBase::ImageResponse ImageResponse;
typedef ImageCaptureBase::ImageType ImageType;
typedef common_utils::FileSystem FileSystem;

client->confirmConnection();

while(true)
{
    clock_t beginFrame = clock();
    vector<ImageRequest> request = {ImageRequest("0", ImageType::Scene),
                                    ImageRequest("1", ImageType::DepthPlanner, true)};
    const vector<ImageResponse> &response = client->simGetImages(request);
    clock_t endFrame = clock();

    deltaTime += endFrame - beginFrame;
    frames ++;

    //if you really want FPS
    if( clockToMilliseconds(deltaTime)>1000.0){ //every second
        frameRate = (double)frames*0.5 +  frameRate*0.5; //more stable
        frames = 0;
        deltaTime -= CLOCKS_PER_SEC;
        averageFrameTimeMilliseconds  = 1000.0/(frameRate==0?0.001:frameRate);

        if(true)
            std::cout<<"FrameTime was:"<<averageFrameTimeMilliseconds<<std::endl;
        else
            std::cout<<"CPU time was:"<<averageFrameTimeMilliseconds<<std::endl;
    }
}

return 0;

}

2. Run python code and monitor output and notice it's a lot faster at 25+ FPS

import setup_path import airsim import time

connect to the AirSim simulator

client = airsim.MultirotorClient() client.confirmConnection() client.enableApiControl(True) client.armDisarm(True)

while True: start_time = time.time() # start time of the loop

get camera images from the car

responses = client.simGetImages([
    airsim.ImageRequest("0", airsim.ImageType.DepthVis),  #depth visualization image
    airsim.ImageRequest("1", airsim.ImageType.DepthPerspective, True)])  #scene vision image in uncompressed RGBA array
print("FPS: ", 1.0 / (time.time() - start_time)) # FPS = 1 / time to process loop```

[jonyMarino]

Hi @MoBaT! Thanks for your report. First of all, I would like to know how you are using the latest commit of AirSim with UE 4.22. You are not getting any error? I thought we are only supporting >= 4.25

[MoBaT]

Hi @MoBaT! Thanks for your report. First of all, I would like to know how you are using the latest commit of AirSim with UE 4.22. You are not getting any error? I thought we are only supporting >= 4.25

I'm so sorry! It was actually 4.25.

[MoBaT]

Has anybody has this issue as well or found a way to get around it and get an fps similar to python?

[xxEoD2242]

Hey @MoBaT, try settings ViewMode: NoDisplay in your settings.json. That increases the FPS rather significantly but you will lose the visualization in Unreal Engine. We have achieved above 30 FPS with this.