Fixed Wing Aircraft

[AOS55]

Fixed Wing Vehicle Integration

I am working on adding a FixedWing aircraft class into Airsim, so far I have been documenting my work under issue #2508 but as suggested by @rajat2004 it would be usefult to move this work to a new issue so as to improve visibility for others to work on.

Work so far

Please see this post for a detailed breakdown of what has been added in my forked Airsim repository under the fixedwing branch. I will aim to keep updates on this issue of new files, classes, methods & functions with a view to write comprehensive documentation as the code begins to get to a state where it can be merged back into the master branch on the main repo.

Since the last update post on #2508 I have been updating the FixedWingLibClient files to enable communication with Ardupilot. A linked issue has been opened within the ArduPilot Repo.

FixedWing Branch State

• Compilable from Visual Studio: 🟢
• Runs alone in UE4: 🟢
• Runs with ArduPilot and Mavlink in UE4 over network: 🔴
• Model validated with real comparison: 🔴

Known issues

• I can't compile the code from source in Visual Studio please see the chain back in #2508 for what we have tried so far. I think it is most likely a mistake I have made in the source code. <-- This is currently the most pressing issue.
• I naievly initially implemented the control surfaces at seperate functions getElevatorSignal() etc. I am beginning to realise as I work through the control implementation this was not a good way to go about this and the program is already well setup to accept a more conventional control vector, i.e. the u vector in state-space form. I am planning to go back and correct this to a 4 row u vector.
• The Core APIs I effectively copied over when setting up the controller files include several aspects that I don't think make sense for a Fixed Wing aircraft, for example commandVelocityHold uses the function moveByLocalVelocity that takes a boolean yaw mode switch and rate as a command these are set as false and 0 in the current implementation. But, moveByLocalVelocity may be better of if rewritten for fixedwing.

Planned Future Work

Initially I am aiming to work to resolve all the known issues and turn all the 🔴 in the state to 🟢. Following from this I will aim to add the following functionality: [ ] Construct a similar Python API as in multirotor <-- hopefully a short term objective [ ] Implement Undercarriage for fixedWingLandings [ ] Deep Reinforcement Learning (DRL) based FW landings <-- main PhD objective, expect to write & publish a paper based on this [ ] Validate model and control on real UAV and FW aircraft, ties in with above.

Motivation

Principally this is for my PhD at the University of Bristol where I am aiming to use a combination of computer vision and Deep Reinforcement Learning to train an autonomous fixedwing agent/aircraft to navigate to and safely land in an unknown location. This is both with and without power and on a range of fixedwing aircraft including fixedWing UAVs, gliders & multiengine light piston aircraft (<5700Kg). Not only this but when discussing AirSim with my colleagues we believe that having a fixedWing simulator capable of highfidelity graphics and SITL that is efficient will be of great benefit.

Ongoing Development

Several users have already been realy helpful (@rajat2004 😀), if anyone has any ideas about how to solve some of the issues I have described above all help is greatly appreciated. In particular if anyone is able to have any success compiling the fixedwing source please let me know as I'm a little stumped 😕 as to why it won't compile.

[rajat2004]

OriginGeopoint goes outside the Vehicles section, along with SimMode, SettingsVersion

[AOS55]

Great thanks, it now looks sort of like it is dropping in but immediately ends on the floor, do you set the fps from the settings.json as the mavlink console says FPS avg=333.33 Im wondering if its running so fast that you cant see it descending. Or do you know of a method to get a log from ardupilot or airsim

[rajat2004]

The 333.33 Hz FPS is the physics rate, not the visual one. FastPhysicsEngine runs at that fixed rate, can be slowed using ClockSpeed. I'm not sure what you mean by dropping immediately, the Alt reported by Mavproxy windows goes to 0? A screen capture would be nice, will try it out soon. AP stores logs in a folder called logs, in the current directory

[AOS55]

Ok that makes sense, yes sorry should have been clearer as the object renders into unreal it looks like the tail comes down and then it lies flat on the ground but that could just be it initializing. I have a screen grab here, great thanks.

I can't see the logs folder is it in the ardupilot directory or the airsim directory, I see mav.tlog but that seems to be a binary or is unreadable with cat mav.tlog.

I also notice that Ardupilot calls APM: Ground start complete in the console do you know what that does or is it just a sensor initialization command?

[rajat2004]

Okay, I think the logs haven't been created since the vehicle isn't armed, you can set LOG_DISARMED to 1, that will generate the logs without arming the vehicle. Also, the logs are binary files, you can explore them using Mision Planner or Mavexplorer.py. See https://ardupilot.org/dev/docs/common-logs.html

Probably sensor initialization, I haven't played too much with Plane so not sure

Also, there seems to be some connection problems, you're using WSL? 1 or 2?

[AOS55]

Cool, got that working added LOG_DISARMED = 1 into airsim_plane.parm, it seems to be starting on the ground in a vertical attitude from the logs: .

Ok sure.

Im on WSL 1

[rajat2004]

You could also uncomment the #if 0 & #endif here, that'll log the raw data sent by AirSim as well in separate messages. It might be better to discuss about the physics and log analysis with the much more knowledgable folks over at AP's Discord server or the forums.

Hmm, haven't had any problems with WSL1 when I tested, maybe something's changed, does it reconnect or stay disconnected?

[AOS55]

Hmm ok, uncommented them now and removed ' as this was throwing a warning, thats a good idea Ill send a message in general. I can't seem to see where the ASM1 log message appears

I think it reconnects not sure though this is what typing messages into MAVExplorer.py gives me:

MAV> 2020-09-01 15:16:08 Throttle failsafe off
2020-09-01 15:16:08 ArduPlane V4.1.0dev (1ecc1618)
2020-09-01 15:16:08 5f2e394f13010c736f29b85d5f312ef0
2020-09-01 15:16:08 Param space used: 260/3840
2020-09-01 15:16:08 RC Protocol: SITL
2020-09-01 15:16:08 New mission
2020-09-01 15:16:08 New rally
2020-09-01 15:16:10 Airspeed 1 calibrated
2020-09-01 15:16:14 GPS 1: detected as u-blox at 230400 baud
2020-09-01 15:16:15 EKF2 IMU0 initial yaw alignment complete
2020-09-01 15:16:15 EKF2 IMU1 initial yaw alignment complete
2020-09-01 15:16:20 EKF2 IMU0 tilt alignment complete
2020-09-01 15:16:20 EKF2 IMU1 tilt alignment complete
2020-09-01 15:16:25 EKF2 IMU0 origin set
2020-09-01 15:16:25 EKF2 IMU1 origin set
2020-09-01 15:16:51 EKF2 IMU0 is using GPS
2020-09-01 15:16:51 EKF2 IMU1 is using GPS

[rajat2004]

@AOS55 Just tried out the latest commits, things look okay to me from the logs, noticed that there were some problems with the logs, opened https://github.com/AOS55/ardupilot/pull/1 which enables the logging of Airsim data Here's a log file with the ASM1, ASM2 messages - https://drive.google.com/file/d/1ZhiCmiXtN6-KAdYzklJlkL6k6HlzI5tN/view?usp=sharing

[AOS55]

Great thanks a lot Ive commited that back to master, I also added support for tla in the message packet and a very simple linear physics implementation for this.

Thanks how did you view the log file, was there a specific command in MAVExplorer.py?

• Ive tried graph ASM1 but it doesn't seem to give any output.

[rajat2004]

Try graph ASM1.Alt, after ., it should autocomplete. MAVExplorer is somewhat strange, if you want, we can discuss about the problem on chat or voice, since it can get messy with logs and all, not particularly well suited on an issue

[AOS55]

Cool that sounds good which program do you want to use/ what works best for you?

[rajat2004]

Sorry, missed the message, Github didn't show notification Discord works?

[AOS55]

Yeah discord is good

[AOS55]

https://discord.gg/xd4PrN

[AOS55]

Hey Ive been looking at debugging some of the flightphysics problems with the FixedWing class I think I've worked out what is wrong with the implementation but not sure how to fix the implementation. The fixedwing physics files in AirLib/include/vehicles/fixedwing/ should construct an aircraft as follows:

• Basic AircraftControls are implemented in ControlSurfaces.hpp but no forces are communicated in this method only the control signal.
• Airplane.hpp contains the main aircraft physics and should apply a wrench to the aircraft
• As in the multirotor, FixedWingPhysicsParamsFactory.hpp inits the params_ object based on the setupframe... defined in AircraftParams.hpp.
• FixedWingPhysicsBody sets the aircraft pose and advances the model through the simulation.

Currently the methods in Airplane.hpp outside of initialize are not being called ideally the class should execute the following methods:

1. Based on state setAoA and setAirspeed and createControls using the method defined in ControlSurfaces.hpp
2. Aerodynamic forces are then calculated using createAeroForces
3. The aerodynamic forces are then applied to the Airplane : PhysicsBodyVertex class using setWrench, I think the setWrench method is currently not being called and the new forces & moments not applied to the vehicle around the loop.

Do you know how to apply forces and torques to a PhysicsBodyVertex child?

I think the createDragVertices method used in the multirotor implementation might make the most sense but this only applies forces offset from the CG rather than a single vertex with forces and moments defined about the CG as is done in Aircraft.hpp?

[rajat2004]

Not exactly sure what's going on, will need to run and try to understand the flow, but from a first glance, there are 2 derived classes from PhysicsBodyVertex, one is Airplane.hpp, and the other's ControlSurface Don't think that's intentional, might give some clues as to why some methods are not being called

[AOS55]

Sure, I think thats from where I initially built the ControlSurface class based on the RotorActuator class from Multirotor. Im not sure if ControlSurface should still be a derived class, it will need to update in the loop. As it receives command signals from the Ardupilot Api and then sends these signals to the Airplane Class in createControls.

Given Control surface just returns control deflection do you think it needs to be a PhysicsBodyVertex derived class. Also looking at the multirotor in further detail the PhysicsBodyVertex::update() function call should be responsible for the setWrench method being used. Perhaps the update() method in Airplane.hpp is never called.

[AOS55]

Hey, Its been a couple of weeks since I put something up here. Ive been looking back over the program. I noticed in MultiRotorPhysicsBody an override is used to override the function, and assigns the forces to the PhysicsBody. I didn't include this as I thought my setWrench method in the Airplane.hpp class was doing this, its quite clearly not.

Do you know if there is a way to get the getWrenchVertex to be used for just a single structure rather than the class templated as a vector. I could change add a new getter into PhysicsBody something like getSingleVertex without a for loop, I don't know if this is a bit of an antipattern however. Would be great to get anyones about how to best restructure this.

[rajat2004]

I haven't looked deeply at this, but you could possibly move the parts in PhysicsBody not suitable for FixedWing to MultiRotorPhysicsBody, which I guess includes this. That'll have the benefit of making the PhysicsBody more generic. The functions called in PhysicsBody::update() anyways are actually implemented in MultiRotorPhysicsBody

[AOS55]

Yes that makes sense I'm not entirely sure how PhysicsBody actually calls getWrench though. I have tried the following implementation calling the same getWrenchVertex, this returns an unreal textbox error stating error at startup: invalid vector subscript. Then goes to this UE4 triggered breakpoint. Any ideas?

[rajat2004]

getDragVertexCount() seems missing, maybe try adding that? But don't think that'll actually affect anything, since the base implementation in PhysicsBody returns 0 which will cause the loop to not run Also try adding some print statements or breakpoints here and there to make sure things are being called correctly

[AOS55]

Thanks yeah I tried playing around with it but doesn't make any difference. Ive reorganized Airplane.hpp slightly. setWrench wasn't an actual vector before I realised this should now be fixed. I've been trying to work through the various base classes surrounding the physicsEngine.

A useful part in FastPhysicsEngine shows how getBodyWrench from PhysicsBody is used. When debugging I keep getting the error window shown below when going through breakpoints is this something you have come across before? Tried pressing load and playing with symbols in visualStudio tools but with no success.

Im also still not sure what is causing the error at startup: invalid vector subscript message from unreal.

[rajat2004]

Nope, haven''t seen that earlier

[AOS55]

Cool np, Ive sent a message over to the unreal forum (which is awaiting mod approval) and the same message to the UE4 subreddit. There might be some more people there who are more familiar with the UE4 side of things.

I've been going through the breakpoints line by line trying to work out the differences between my method and multirotor and also where the differences lie. The following questions/potential solutions have come up Im going to keep investigating further:

• MultirotorPhysicsBody.hpp has the rotors_ struct as a vector of instances, whilst FixedWingPhysicsBody.hpp Im wondering if FastPhysicsEngine expects the returned wrench to be a vector, I can't see anything in the class's method to suggest that to be true though. If anything it should be a wrench as is passed, it may not be in FastPhysicsEngine at all.

• Within MultirotorPawnSimApi.cpp the rotor_count variable to control is passed with wrenchVertexCount, this causes the wrenchVertexCount method from MultirotorPhysicsBody.hpp to run (not sure if it overrides into PhysicsBody as well at this point). This combines the thrust and controllers into one object. On the FW airplane however the control surfaces impart no force directly they are not a wrench instead when the control is deflected the airplane : PhysicsBodyVertex class runs and the new control input sets the force change in the loop. control_count is found using a method that just gets an unsigned int from params. This is not coupled to the forces or wrench and I'm not sure where it would be but I may have upset the way wrench is called. Running through the code with breakpoints I found that resetImplementation is an important call for setWrench and as updatableObject is a base class of several methods is how a lot of the virtual functions get overriten in the initial instance. So as long as uint wrenchVertexCount != 0 the wrenchVertexCount method will run. It may be however the UE4 vector error comes up because airplane_ is a class and not a vector of class instances as is the case for rotors_.

• Finally, I might try also try working back from where UE4 places the error i.e. in AirSimGameMode.cpp the problem could be to do with the FixedWingActor as implemented through unreal I'm not sure where to trace this back, I have FixedWingActor but Im not sure where the spawned pawn/actor gets updated. Im guessing UpdatableObject is somehow linked to this but not sure where this and the API links with an unreal method.

Sorry bit of a long post, mainly put up to keep track myself, are there any other thoughts of where might be useful to look?

[AOS55]

Update:

Managed to solve the UE4 error with some assistance from one of my colleagues. The controlSurfaces were not being constructed in the aircraft and a function was called to a vector<struct> that caused the array problem. Have a loss of connection with Airsim and unreal now due to [2020.09.21-17.44.45:574][215]LogTemp: Error: Linear velocity had NaN! error being returned in the command line. Not quite sure what is causing it yet but certainly making some more progress.

[AOS55]

AirSim Physics

I might open this up in a seperate issue thought I'd put it here first. Does anyone know the orientation of the following outputs from kinematics. In the current fixedwing commit the values of aero forces are becomming very large and causing an overflow/nan error. I have used the following to define each input:

• Aircraft orientation as quaternion kinematics_->getState().pose.orientation the body axis defined relative to earth axis horizon of form [s + q1i + q2j + q3k].
• Aircraft position as displacement from orgin kinematics_->getState().pose.position [xe, ye, ze].
• Linear Velocity along body axis kinematics_->getState().twist.linear [u, v, w].
• Angular Velocity along body axis kinematics_->getState().twist.angular [p, q, r].
• Linear acceleration along body axis kinematics_->getState().accelerations.linear [axb, ayb, azb].
• Angular acceleration along body axis angular_acceleration = kinematics_->getState().accelerations.angular [pdot, qdot, rdot].
• To calculate the aircraft's orientation in the body axis the function toEuler converts a quaternion to an euler vector representation of the aircrafts orientation. This should contain the angles [roll, pitch, yaw].
• The wind axis orientation should then be found from the dot product between the velocity vector and the orientation vector calculated prior, the function angleBetweenVectors is used to accomplish this. This should the aircrafts aero angles as input [alpha, beta, psi] (note psi the zw to zb axis angle is unused assume symmetric in this sense).

I think the most likely cause of the problem is either:

• Aero angles are defined with wrong reference i.e. pose/twist/acceleration is relative to earth axis not body axis.
• Incorrect implementation of controls in ardupilot
• Incorrect coefficients used I have assumed pose is setup in radians NOT degrees

[rajat2004]

Yeah, probably might be a good idea to open a new issue. I myself mostly won't be able to take a look at all this anytime soon, but hopefully others will help, There's also another PR #2956 which might be of interest, that one also fixes an FPE which happens with extreme values with AirSim+ArduPilot Rover

ArduPilot side, as you probably already have been doing, could capture some logs and maybe ask on the Discord server also.

[AOS55]

Thanks, I added #2956 into the distro but it doesn't seem to have had any effect, if anything it appears its accelerating far too much and has too much dynamic pressure i.e. runaway, (perhaps not enough drag, but L/D seems to be 5 which is pretty normal).

As I've been debugging stuff further today I think the problem may actually be to do with ground collision in fastPhysicsEngine in particular getKinematicsOnCollision, Ill keep looking at it but it seems to cause huge linear and angular accelerations on the body resulting in the values blowing up. Not sure why yet?

[AOS55]

I've been working through debugging the physcis with the fixedwing vehicle. I have debugged most of the issues that were present in the aerodynamics side of things. I had a couple of dimensionless coefficient mistakes and was returning the aeroforce to fastPhysicsEngine in wind axis rather than body axis. This is now fixed and the force is in body axis. I still have the variables blowing up in places however, I'm beggininig to think this might be in the fastPhysicsEngine itself.

The engine uses Verlet Integration to solve the problem which appears to scale with O(delta_t^2) error. Most fixedwing simulations I have seen in the past tend to solve the state-space system of equations with the euler method instead, which notably scales with O(delta_t) error. I'm not sure about the time complexity comparison of each method without investigating further maybe this is why its called fastPhysicsEngine?

Does anyone know how to change the value of dt to a smaller increment as Im wondering if the simulation problem is actually a numerical integration error.

[seanmcleod]

@AOS55 your other general option is to make use of an external simulation engine like JSBSim for the aerodynamics and physics modeling rather than re-implementing most of it within AirSim's physics engine. Then use the pose data from JSBSim to drive AirSim in computer vision mode in order to retrieve high-fidelity camera images for computer vision machine learning etc.

I sent you an email a couple of days ago to your university email address regarding this option with a bit more detail, may have ended up in your spam folder.

[AOS55]

@seanmcleod thanks your email had indeed gone to my spam inbox, response now in your inbox. I think this certainly a good idea and I'll aim take a closer look over the next week.

[jonyMarino]

@madratman @saihv @ironclownfish @jonyMarino Do you think having another (more chat & voice type) communication platform will be worthwhile? This will definitely increase community interaction, and will probably reduce support issues. Just wanted to know your opinions, and definitely don't want to increase your (already heavy) workload too much. Looking forward to hearing from you!

Hi @rajat2004! It is something we are already discussing. In my opinion is something necessary. Will update.

[tensarflow]

Wow really awesome stuff you guys are doing there, appreciate it! I was using gazebo for fixed wing UAV simulations occasionally for several years. Since I would like to do some realtime computer vision I would like to have a photorealistic environment within the simulation too. That's when I came across this thread. It is not really clear though, what the current status is since your first comment on this thread. Could you guys maybe write a quick recap of what we can use on your branch and how? Unfortunately I don't really have the time that I can invest by contributing here (which I very much would like), since it's more of a hobby. That's why I am also looking for a more quick-and-dirty kind of solution if your branch is not quite usable. So in that direction something came to my mind :) In gazebo fixed wing UAVs are very well supported. At least it satisfies my needs. How about just running the physics stuff on gazebo, then somehow subscribing to the position/direction/velocity etc. messages of Gazebo inside Unreal Engine and only updating that information. Putting the 3D Model of a UAV would be pretty much optional in that case. A flying camera would do it for me :) What do you think? Does that make sense?

[seanmcleod]

How about just running the physics stuff on gazebo, then somehow subscribing to the position/direction/velocity etc. messages of Gazebo inside Unreal Engine and only updating that information.

I mentioned that option, although I suggested using JSBSim for running the physics/aerodynamics, see my comment a couple of comments back - https://github.com/microsoft/AirSim/issues/2955#issuecomment-708978788

[xxEoD2242]

I've been working through debugging the physcis with the fixedwing vehicle. I have debugged most of the issues that were present in the aerodynamics side of things. I had a couple of dimensionless coefficient mistakes and was returning the aeroforce to fastPhysicsEngine in wind axis rather than body axis. This is now fixed and the force is in body axis. I still have the variables blowing up in places however, I'm beggininig to think this might be in the fastPhysicsEngine itself.

The engine uses Verlet Integration to solve the problem which appears to scale with O(delta_t^2) error. Most fixedwing simulations I have seen in the past tend to solve the state-space system of equations with the euler method instead, which notably scales with O(delta_t) error. I'm not sure about the time complexity comparison of each method without investigating further maybe this is why its called fastPhysicsEngine?

Does anyone know how to change the value of dt to a smaller increment as Im wondering if the simulation problem is actually a numerical integration error.

Have you thought about changing this value in the Fast Physics Engine? I linked to the place where dt is instantiated. I'd be really careful with this, as the time step calculations for this process may be heavily dependent on a sufficient timestep. Have you thought about taking the underpinnings of JBSim and modifying the FastPhysicsEngine to be something like kindFastPhysicsEngine with more realism?

https://github.com/microsoft/AirSim/blob/d59ceb7f63878f5e087ea802d603ba0fd282ff56/AirLib/include/physics/FastPhysicsEngine.hpp#L76

Myself and a team are looking to implement a fixed wing drone for a drone competition but we also want to update the physics models to be a bit more realistic for dynamics information so that we can implement our state-space models more efficiently then writing them externally in Python.

[AOS55]

Hi @xxEoD2242 thanks for this. I think a JSBSim physics engine is what we should be heading towards it makes a lot of sense and prevents us reinventing the wheel. I have been working on a python implementation over the last month. But having it native to airsim in C++ would be ideal. The problem I've begun to realise over the past couple of days is that by pose forwarding JSBSim to airsim in CV-mode the collisions cannot be easily calculated from unreal as there is no physical mesh geometry like you get in the multirotor or car pawn.

I'd certainly be keen to work together on this though. A native C++ framework is certainly the way to go long term, with a python API similar to multirotor. I think it might be best to let JSBSim deal with all of the fixedwing physics. JSBSim also has a ground interaction component, but I'm not entirely sure how it works yet, the aircraft I have defined so far does not have any ground-reactions in the aircraft XML file.

[xxEoD2242]

@AOS55 That makes sense. I think the appropriate solution would be to implement JBSim natively in the build process with AirSim and have the selection of JBSim for dynamics information. I work with Purdue University researchers a lot so I'll see if there would be interest from the faculty there in exploring how we would achieve this. Signal routing through the same FastPhysicsEngine routes would be my biggest concern, especially from a memory standpoint to ensure that AirSim stays as optimized as possible. If we decide to go down this route, I'll let you know and possibly fork your branch since you have so much done.

[AOS55]

The documentation and tests leave a little to be desired at the moment but you might find my FixedWing-AirSim repo useful, it just implements the pose forwarding with CV mode described above and then runs JSBSim for the flight physics.

The key classes are in jsbsim_simulator.py

[xxEoD2242]

Awesome. Thanks!

I think there is a desire to implement the physical drone as well and I have a teammate who is working on creating custom environments and scenarios. If we can figure out the blueprints and meshes correctly, we will try to implement that as well, which should help with the issue you are having in CV mode.

[AOS55]

@AOS55 That makes sense. I think the appropriate solution would be to implement JBSim natively in the build process with AirSim and have the selection of JBSim for dynamics information. I work with Purdue University researchers a lot so I'll see if there would be interest from the faculty there in exploring how we would achieve this. Signal routing through the same FastPhysicsEngine routes would be my biggest concern, especially from a memory standpoint to ensure that AirSim stays as optimized as possible. If we decide to go down this route, I'll let you know and possibly fork your branch since you have so much done.

Great that would be really useful and its always useful to get others opinion on how to best implement this. Yes I'm not sure on the difference in efficiency JSBSim presents @seanmcleod may be able to help there.

Awesome. Thanks!

I think there is a desire to implement the physical drone as well and I have a teammate who is working on creating custom environments and scenarios. If we can figure out the blueprints and meshes correctly, we will try to implement that as well, which should help with the issue you are having in CV mode.

Great also would be very helpful

[seanmcleod]

in CV-mode the collisions cannot be easily calculated from unreal as there is no physical mesh geometry like you get in the multirotor or car pawn.

Yep, I see a feature request was raised for collision detection in CV-mode in Aug 2020 - https://github.com/Microsoft/AirSim/issues/1565

So for now you would need to make use of JSBSim's collision detection via it's ground callback mechanism etc.

I had originally suggested the JSBSim pose forwarding idea to Airsim in CV mode as a quick and fairly easy mechanism to implement. Linking JSBSim into AirSim and having some integration and switching mechanism with Airsim's physics engine is definitely doable but a lot more work.

Would the Airsim repo maintainers be interested in incorporating JSBSim in this way? Or would you have to maintain your own fork?

[AOS55]

Hey a couple of questions for the airsim maintainers and perhaps JSBSim too. I have been looking at integrating JSBSim into AirSim over the past week, using the car and multirotor pawns as a base. Im not sure how best to structure the code to get it to work with airsim.

As @seanmcleod stated above jsbsim has a collision model built in for a STRUCTURE and BOGEY (undercarriage) element. There is a wrapper class in JSBSim FGFDMExec that is used to instantiate a jsbsim object and then the simulation can be advanced with run and properties accessed and set.

Im not sure how to connect the class of JSBSim to an Airsim UE4 pawn object necessary for physics and then send back the terrain information (this can probably come later though). I believe I need to call the wrapper class on a class derived on AirSim's UpdateableObject base class. I have created a simple class JSBSimPhysicsEngine that should serve this purpose. I'm not sure how to call this method with a pawn, I initially followed a similar structure to the car api but did not realize it used Unreals inbuilt physics engine. I'm not sure where multirotor builds its FastPhysicsEngine model either.

I believe the model needs to do the following and I have started making classes for each of these but am unsure of the interaction:

• Provide a method to instantiate an AirSim Api base & derived plane.
• Create a Pawn representing the physical body in UE4 and capable of receiving collision information
• Create a Pawn API to communicate with the pawn
• Create a simmode to setup the pawn and link ue4 blueprints to the vehicle.
• Instantiate JSBSim.

Does anyone have any ideas about the best way to link the JSBSim physics library with the airsim UE4 pawn library. The collision information from UE4 would ideally be sent to JSBSim's ground callback mechanics or we could do it in UE4 and set the acceleration/state. Also not sure if the JSBSim python instance can be used this way I think the best way to interface with the running program will be to setup a pythonclient the same as airsim's multirotor and fixedwing not sure what the way to do that will be.

Are there any UML type diagrams for AirSim the maintainers or microsoft have they might be able to publish to show the derived class interaction for the Car and the Multirotor this might make the task of adding a new vehicle class easier in the future too. Thanks for everyone's help.

[AOS55]

Following up from my previous post, I have been working on the JSBSim addition today. Im still not 100% how information is supposed to be sent from the Pawn to the API. In particular I have 2 structs JSBSimControls and JSBSimState.

I believe the interaction between JSBSim's wrapper FGFDMExec should be something similar to the block diagram shown below: With JSBSimPawnSimApi responsible for getting state information from JSBSim along with calling JSBSim update (via run) and instantiating JSBSim. The class should also pass information to JSBSimPawn to set the aircraft's pose in UE4. Information from JSBSimPawnSim should then be passed to JSBSimApiBase used as the base for a plane specific api (JSBSimPlaneApi) in here all the autopilot functions can be created to control the aircraft and update the JSBSim aircraft. I have the following outstanding questions and it would great to get anyones thoughts:

• How would you send state & control information in the architecture I have described above is it via the control and state structs, in Multirotor and car these are consts so I am not sure how to update them if following a similar architecture.
• How would you go about setting environment and kinematic variables for the api, I'm guessing these are needed to setup the sensor_factory which I believe are required will these just come from initial conditions, which can come from JSBSim in turn.
• To deal with collisions what do you think is the best way to do this? AirSim typically returns CollisionInfo, is the information in a form that can be quite easily used JSBSim's ground callback mechanism or would we be better of having a ground reaction method that provides alters the returned state following a collision separately?

[AOS55]

If anyone has any ideas on what Im missing in #3181 that would be really useful I am trying to sort out linking the static JSBSim library to AirSim but I am having problems with visual studio.

[jonyMarino]

Hi @AOS55! Thanks for the amazing work you are driving here! I think we can simplify something in your design. I don't know if there is a simple controller for JSBSim (like SimpleFlight), but it is integrated with the autopilots. So it would seem like a good idea to leave the control on the JSBSim side and leave AirSim alone with perception and reporting collisions. What do you all think?

[AOS55]

Hi @jonyMarino thanks a lot. Yes there is xml files for a lot of the aircraft that sets the gains for controllers defined inside JSBSim, for example the c172x ap. I have also found it useful to directly control the aircraft using a python script which I did for the x8. I expect to use the same script to control the aircraft but with collisions also being calculated by airsim, i.e. update_airsim should also return a collision value and the connection between airsim and jsbsim will happen in the airsim C++ side rather than in the python script, this should be what JSBSimPhysicsEngine.

[jonyMarino]

Great! This development would have an incredible impact if it could be used for multirotors as well. Why is it that I don't see much work done in JSBSim for that? It is not the best option?

[AOS55]

Thanks, yes certainly there are quite a few useful physics simulations included with JSBSim too such as heavier than air powered rotorcraft (helicopters), airships etc that I haven't looked at but I'm sure others would find useful. My best guess as to why multi rotors haven't been extensively developed is probably age? I think JSBSim started in the late 90s or early 2000s and they just weren't considered significantly at this time except for rc aircraft @seanmcleod may know more?

[seanmcleod]

In terms of JSBSim and multirotors take a look at the following - https://github.com/JSBSim-Team/jsbsim/issues/333