What is the right way to compile airsim plugins in Ubuntu?

[Amigoshan]

I would like to make some changes to airsim plugins. Currently I'm using ubuntu16.04, latest airsim on github, and Unreal Engine 4.17.2. I'm quite confused about the compiling things.

I have observed the following things:

1. When using setup.sh and build.sh in Airsim, the files under Unreal/Plugins won't be compiled. They will be copied to Unreal/Environment/Blocks project. -- So I guess the plugins are compiled with the Unreal project. If I would like to change the plugins, I need to change the files within the project's plugins folder, instead of the files under Airsim.

2. I created a new c++ project named 'testAirsimPlugins' in Unreal Engine, copied the plugins folder and modified the .uproject file. In this way, the drone model 'BP_FlyingPawn' could be successfully added to the environment. The problem is when I changed files in the plugins folder and hit 'Compile' button in Unreal, nothing happened. I also tried run 'make' in the command window under project directory, it won't build the plugins either. But if I run 'make testAirsimPlugins', the plugins seemed to be compiled, together with a whole bunch of other modules, which took really long time.

I feel that I'm not doing it right way.

Also there's other problems I'm not quite understand. After I drag 'BP_FlyingPawn' into the new environment, it is not functioned like the one in block example. The simulation windows for the camera could not be shown (normally they will pop up when hitting key '0'/ '1'/ '2'/ '3'). And the drone is not ready to be connected through RPC. Am I missing something here?

--- update--- After I setting Window - World Settings - GameMode Override to AirSimGameMode, the drone functions well now. Still need some instructions on plugins compilation.

Thanks!

[sytelus]

Your hunch is correct. There are basically two compilation systems here. One is for AirLib which is pure C++11 code without any Unreal dependency. When you do build.sh, that's what gets compiled. Other is Unreal plugin which requires Unreal's build system. For the very first time, Unreal will detect that plugin needs compilation so it will do it. But unfortunately it has no smartness to detect that plugin code changed later, so let's recompile it :(. This is precisely, why we have clean.sh which removes all temp folder in Unreal project to force the build system to re-compile the plugin.

If you are doing Unreal development in LInux, it's going to be very miserable experience with long compile times and giant logs to dig through. I think Unreal supports running applications on Linux but if you are developing then you better switch to Windows.

The way I've set up my workflow is that I use Visual Studio on Windows and make changes in my custom Unreal project directly. You just have to press F5 and test the change. When everything works, I copy Plugins folder back to AirSim git repo.

When AirSim/Plugins have update, I replace Myproject/Plugins with AirSim/Plugins and then run clean.bat followed by GenerateProjectFiles.bat (both available in Blocks sample project in repo).

For the 2nd question, AirSim actually detects if you had any vehicle pawns in the scene already. If there is then it disables auto-create of vehicle pawn (see this code). This was to support multiple vehicles scenario which unfortunately is broken currently and we will fix soon.

PS: It's good idea to file two different issues for unrelated stuff.

[Amigoshan]

@sytelus Thanks for your clear explanation. I've successfully setup the environment. I would like to ask something about the code structure. I would like to add an API for gimbal control. I find there's already an API for setting the angle: setCameraOrientation. But this function sets the angle instantly. What I need is an API with dynamics such as moveByVelocity and moveByAngle. I've been reading the code for a few days. From my understanding, the MultiRotorConnector update the drone's pose in the updateRenderState and updateRendering functions. FastPhysicsEngine takes care of the drone dynamics, and update the drone's position, velocity and acceleration. This diagram shows my understanding of the airsim project.

What confuses me is I couldn't find the relation between the MultiRotorAPI and PhysicsEngine. I see the MultiRotorAPI calls DroneControllerBase. But how does the controller interact with the PhysicsEngine? Where does the controller set the velocity of the PhysicsBody after the moveByVelocity has been called?

@sytelus Could you give a hint on this? Thanks a lot!

[Amigoshan]

After a few days of digging into the code, I have to complain that adding a gimbal control API is not as easy as I thought. This is an updated diagram of my understanding of the code structure.

You have to modify almost every file in the above structure.

1. add API function in MultiRotorRpcLibServer, MultiRotorAPI, DroneControllerBase.
2. add goal-setting support for camera gimbal in OffboardAPI, IGoal and all kinds of controller.
3. modify the kinematics for camera in PhysicsBody, FastPhysicsEngine, MultiRotor.
4. set the camera pose for unreal engine in MultiRotorConnector, VehiclePawnWrapper, PIPCamera.

I may not understand it well, I find some of the code are not in good structure. For example, some of the controllers are derived from IGoal. When I add a new virtual function for querying camera's goal pose, I have to change all those controllers as well. I would like to reuse the AngleRateController for the gimbal control but finally I give up because it is tightly coupled with the OffboardApi.

[sytelus]

I've added gimble stabilization using settings (so camera's roll, pitch and or yaw axis is stabilized). Please see this commit: https://github.com/Microsoft/AirSim/commit/95853ad9f18b6d1d45103ec60d3bd534ac95e57b

There is also API for camera orientation: https://github.com/Microsoft/AirSim/blob/master/docs/image_apis.md#camera-orientation-gimble

If you need to have velocity to change camera orientation then you can easily do it through Python API by changing orientation at some time interval. You won't need new API for that. If you do want to add API then you can use setCameraOrientation as template and make very similar changes to create new API setCameraOrientationVelocity. The implementation of API will simply change orientation in every update() or Tick() call. However I rather not add this API because many real world system don't have it. So better to just use setCameraOrientation from Python or C++ client.

You are correct about complexity in adding new API. I'm actually working on new design layout which will hopefully cut down some of the complexity but this will take time to get implemented.

[Amigoshan]

Thanks for your reply. @sytelus You described an easier way than I did. In order to reuse the PID controller, I took a long way deep into the firmware. This makes things more complicated. The following picture shows the functions and variables I modified.

Yes, we can simply write a controller in the MultiRotorConnector, that would be easier.

[jameswalmsley]

@Amigoshan Just found a convenient way to recompile under Linux without having to reload and restart the UE4Editor. Its probably not as quick and easy as Windows but heres what I do:

./build.sh # from AirSim dir.
# Copy code to plugins dir, if not using blocks

In UE4Editor click Window -> Developer Tools -> Modules Click on recompile next to AirSim...

[jameswalmsley]

Perhaps this can be added to the Linux documentation as this really helps.

[madratman]

closing as this is old. we'll a doc revamp for all issues labeled with doc-update