search

JSBSim-Team / jsbsim

An open source flight dynamics & control software library
GNU Lesser General Public License v2.1
1.27k stars 435 forks source link

Property explanations #399

Closed JDatPNW closed 3 years ago

JDatPNW commented 3 years ago

Hello! I have been working a bit with XPlane11 recently and wanted to see if JSBSim would work better, as there are features that I prefer, such as stepping physics myself and my own speed, and directly interfacing with the sim - not having to send UDP packets for it.

I am by no means a aviation specialist (I am actually very new to it), and am therefore having trouble understanding the properties. I am currently using the c172 model and I printed out all properties available to me. For XPlane the DREFs (which are basically XPlanes Properties) are listed in their wiki, which makes it easier for someone like me to understand them: DREFS, but the printed list from JSBSim is a bit too technical for me to understand. Is there a list with short explanations?

small extract from the Properties (randomly copied):

ic/vw-east-fps (R)
ic/vw-down-fps (R)
ic/vw-mag-fps (R)
ic/vw-dir-deg (RW)
ic/roc-fps (RW)
ic/u-fps (RW)
ic/v-fps (RW)
ic/w-fps (RW)
ic/vn-fps (RW)
ic/ve-fps (RW)
ic/vd-fps (RW)
ic/gamma-rad (RW)
ic/alpha-rad (RW)
ic/theta-rad (RW)
ic/beta-rad (RW)
ic/phi-rad (RW)
ic/psi-true-rad (RW)
ic/lat-gc-rad (RW)
ic/long-gc-rad (RW)
ic/p-rad_sec (RW)
ic/q-rad_sec (RW)
ic/r-rad_sec (RW)
ic/lat-geod-rad (RW)
ic/lat-geod-deg (RW)
ic/geod-alt-ft (R)
ic/targetNlf (RW)

My plan is to reset the plane (mid air with a velocity, going straight, or something like that) and to then control it per physics step using my code. I would want to control the stick and pedals (or ailerons, elevator and the rudder) at each timestep Receive the:

at each timestep And then when resetting set the same values to a desired state, lets say flying straight ahead for example.

I have somewhat figured out how to do this in terms of writing the code, but I am not sure which properties I need to set. (fdm.set_property_value and fdm.get_property_value)

On a secondary note, I wanted to ask if there is a python documentation, and if I can reset the plane with fdm.run_ic or fdm.reset_to_initial_conditions without the printout:

  Mass Properties Report (English units: lbf, in, slug-ft^2)
                                      Weight    CG-X    CG-Y    CG-Z         Ixx         Iyy         Izz         Ixy         Ixz         Iyz
    Base Vehicle                      1620.0    39.0     0.0    36.5       948.0      1346.0      1967.0        -0.0         0.0        -0.0
0   name                               180.0    36.0   -14.0    24.0         0.0         0.0         0.0        -0.0         0.0        -0.0
1   name                               180.0    36.0    14.0    24.0         0.0         0.0         0.0        -0.0         0.0        -0.0
2   name                               120.0    95.0     0.0    24.0         0.0         0.0         0.0        -0.0         0.0        -0.0
0   Fuel                                 168      48    -112    59.4           0           0           0
1   Fuel                                 168      48     112    59.4           0           0           0

    Total:                            2436.0    42.6     0.0    37.2      1926.9      1481.1      2973.1         0.0         1.6         0.0

End of vehicle configuration loading.
-------------------------------------------------------------------------------

Thanks! JD

seanmcleod commented 3 years ago

I'm not aware of any comprehensive documentation of every property along the lines of XPlane's DREFs.

There are some classes that like FGInitialCondition that have documentation for the properties they expose, e.g.

https://github.com/JSBSim-Team/jsbsim/blob/518f2a74d1a5db65de3986816d3e6cd90521585e/src/initialization/FGInitialCondition.h#L171-L181

These will then also appear in the generated DOxygen documentation that gets generated automatically, see - https://jsbsim-team.github.io/jsbsim/classJSBSim_1_1FGInitialCondition.html for example.

However this documentation will be scattered across the various classes, so you would need to perform a search on the property's name to find it.

Which is what I typically do, i.e. search through the source code for the property name. Which may turn up a hit in some documentation in a header file like the example above, or it may only turn up a hit for the actual source code setup of the property like the following example atmosphere/psiw-rad with some surrounding comments and the names of the methods potentially giving you a better idea of what the property represents.

https://github.com/JSBSim-Team/jsbsim/blob/eccfd64bbe09fc209c399bead031a8ccc5873118/src/models/atmosphere/FGWinds.cpp#L496-L497

I am by no means a aviation specialist (I am actually very new to it), and am therefore having trouble understanding the properties.

What I'd suggest you do is get at least somewhat familiar with a couple of basic concepts, e.g.

https://jsbsim-team.github.io/jsbsim-reference-manual/mypages/user-manual-frames-of-reference/

https://jsbsim-team.github.io/jsbsim-reference-manual/mypages/user-manual-units/

https://jsbsim-team.github.io/jsbsim-reference-manual/mypages/user-manual-flight-control-and-system-modelling/

Attitude angles - [theta, phi, psi] Angular rates in body axes - [p, q, r] Velocity in body axes - [u, v, w] Angles between body frame and wind frame - [alpha, beta] Flight path angle - [gamma]

Also the 'original' JSBSim manual - http://jsbsim.sourceforge.net/JSBSimReferenceManual.pdf

Given that knowledge you can use it to 'decode' a lot of the properties without or before you actually search the source code to double-check, e.g.

ic/q-rad_sec (RW)

ic - initial condition q - pitch rate in body axis rad_sec - units radians/sec RW - read write

seanmcleod commented 3 years ago

@JDatPNW out of interest are you looking into using JSBSim with your QPlane?

In terms of reinforcement learning and the use of JSBSim I've come across a couple of examples.

A DARPA program recently which had 8 teams designing AI agents to perform dog fights against each other with the winning team then fighting against a human pilot. They used JSBSim and the F-16 model in particular.

https://tech.slashdot.org/story/20/08/20/2130216/ai-claims-flawless-victory-going-undefeated-in-digital-dogfight-with-human-fighter-pilot

And a Q&A session with the winning team - https://www.youtube.com/watch?v=uxxkG4uKThY with a number of questions regarding their reinforcement learning model for their agent.

In terms of RL being used in conjunction with JSBSim here is another example:

https://github.com/Gor-Ren/gym-jsbsim

Created as part of Gordon Rennie’s MSc.

https://drive.google.com/file/d/1jPLG-OYcPiffh4ZAWW1N1__4l68jh-G_/view

JDatPNW commented 3 years ago

Hi @seanmcleod, thanks for all the great pointers! I have already skimmed over the thesis by Go-Ren that you mentioned and I saw the dogfight video, but not the interview, which could be really helpful in setting up the RL environment (QPlane), which I am indeed working on. In the process I also found a second thesis here (can be found in the thesis folder), which was done by forking Go-Ran's repo and adding to it. I will try to look at the links you provided sometime this week, and will then get back to you! I really appreciate the advice! Best, JD

JDatPNW commented 3 years ago

Alright! So I have put together something that works without crashing, but it might not yet be fully correct yet. I used Properties fit my DREFs to the best of my knowledge after reading through the docs you sent me, and a good amount of them seem to be right at least. A few I am not perfectly sure about yet (for example the local acceleration) . If someone wants to crosscheck them, feel free to do so :D I will try to compare the XPlane ones to the JSB ones a bit closer tomorrow, and I will also look a bit closer at some of my set and set(ic/...) calls, since I have not yet looked at it in render yet (if the reset does what it should, if the controls do what they should and so on)

seanmcleod commented 3 years ago

If someone wants to crosscheck them,

Where?

JDatPNW commented 3 years ago

I have pushed it to my QPlane repo. It is located here - Sorry I should have made that clear. The XPlane counterpart is here JD

JDatPNW commented 3 years ago

Hi @seanmcleod Thanks for your help so far, it is much appreciated! :) 👍

I have been working quite a lot on this project this week, and it is working out quite well for the most part. But there are a few things that I think are not working how I want them to work yet.

I am trying to get to a point where the XPlane and the JSBSim environments are "identical" and interchangeable (which is why some of these properties are used over others, because XPlane might not have the perfect fit in some cases, e.g. why I am setting local velocities in the ic, because from what I could find, XPlane onlt allows for local velocities to be set). I know the physics are not perfectly identical, but I want to try to at least manage to make all inputs and outputs identical (in the sense that they show the same properties in the same units and or range) and then see if I can train a RL agent that can generalize between the two.

Thanks a lot for helping me with this! Best, JD

seanmcleod commented 3 years ago

In terms of ic velocities causing the plane to go 'crazy' well without knowing what sort of velocities and attitude that are causing these it's difficult to know if they're unexpected or not. Also depends on what you mean by 'crazy'. For example maybe you're starting the plane in such a way that it immediately enters a stall/spin.

What do you mean by "acceleration on the local frame"? Does local refer to the aircraft's body axes, or do you mean a local NED frame?

In terms of control inputs I'd suggest that your code sets the fcs/xxx-cmd-norm properties. They're effectively what the pilot would 'command'. For simple aircraft like the C-172 these pretty much get mapped directly to control positions only with the addition of any potential trim commands.

For more complex aircraft with an FCS, e.g. an F-16 the pilot's command will be an input into the FCS which will have some command law, e.g. g-command such that the pilot's command is mapped to a specific g-command and the FCS via control logic, feedback loops etc. will drive the control positions to achieve the particular command.

When I made contact with one of the developers of the winning alpha-dog trials I asked what their RL controlled and he said they basically controlled the fcs/xxx-cmd-norm properties, i.e. their RL agent flew the aircraft through the FCS model implemented in the standard F-16 model in JSBSim.

JDatPNW commented 3 years ago

Hi! Thanks for the fast response! :) The ic that I was running are:

        self.fdm.set_property_value("ic/lat-gc-deg", 35.126)  # Latitude initial condition in degrees
        self.fdm.set_property_value("ic/long-gc-deg", 126.809)  # Longitude initial condition in degrees
        self.fdm.set_property_value("ic/h-sl-ft", 6000)  # Height above sea level initial condition in feet

        self.fdm.set_property_value("ic/theta-deg", -25)  # Pitch angle initial condition in degrees
        self.fdm.set_property_value("ic/phi-deg", -45)  # Roll angle initial condition in degrees
        self.fdm.set_property_value("ic/psi-true-deg", 0)  # Heading angle initial condition in degrees

        self.fdm.set_property_value("ic/ve-fps", 0 * self.msToFs)  # Local frame y-axis (east) velocity initial condition in feet/second
        self.fdm.set_property_value("ic/vd-fps", 25 * self.msToFs)  # Local frame z-axis (down) velocity initial condition in feet/second
        self.fdm.set_property_value("ic/vn-fps", -55 * self.msToFs)  # Local frame x-axis (north) velocity initial condition in feet/second
        self.fdm.set_property_value("propulsion/refuel", True)  # refules the plane?
        self.fdm.set_property_value("propulsion/active_engine", True)  # starts the engine?
        self.fdm.set_property_value("propulsion/set-running", 0)  # starts the engine?

This causes it to loop. As I am writing this I noticed that I set ic/vn-fps = -55which works with XPlane, I will try +55 and edit this post with the results. What I mean by local is relative to N/E/S/W and not related to the planes position. Thanks! JD

JDatPNW commented 3 years ago

Ok, I have found a few mistakes that I made (to the best of my knowledge, where axis were inverted between XPlane and JSBSim) One last thing that I am looking for (I think) is essentially the acceleration equivalent to velocities/v-east-fps. So basically acceleration/a-east-fps2. I don't seem to find that. I found Nx, Nz, Ny, but I don't really know what they are. Once I have all that done, I would try o create a side by side comparison of the two and upload and share the results. I would just spawn them and let them glide without any inputs from my end to compare the results.

bcoconni commented 3 years ago

@JDatPNW the initial values you are issuing for the pitch (-25 deg) and roll angles (-45 deg) really seem huge - whatever their signs are. These are initializing the aircraft nose down (25 degrees below the horizon) with a bank of 45 deg to the left. Are you sure you want your aircraft initialized in that position ?

In parallel you are setting velocities in the NED frame but the sign of the north component of the velocity seems wrong: the aircraft is heading north (ic/psi-true-deg is 0) but the velocity is pointing to the south (ic/vn-fps is negative). In other words, you are instructing your aircraft to move astern.

Also it seems that you have computed the north and down velocities from trigonometry but you could save yourself the trouble of these computations by setting the velocity in the body frame : setting ic/u-fps to 60 should give you the same result (but you'd need to remove the setting of ic/vn-fps, ic/ve-fps and ic/vd-fps).

The 3 components of acceleration in the body frame can be read from the properties accelerations/udot-rad_sec2 (X-axis, positive is forward), accelerations/vdot-rad_sec2 (Y-axis, positive is starboard) and accelerations/wdot-rad_sec2 (Z-axis, positive is down). They are basically the accelerations that the pilot and the aircraft are submitted to.

As a final note, rather than calling the methods get_property_value or set_property_value, you could use FGFDMExec dictionary-like feature. Properties can be read with fdm['property/name']. The expression is shorter and has the added benefit of checking if the property already exists and sending an exception would that not be the case.

  vn = self['ic/vn-fps']  # instead of fdm.get_property_value('ic/vn-fps')

Properties can be written with fdm['property/name'] = value. There again the expression is more compact for better legibility.

  self['ic/vn-fps'] = 55.  # instead of fdm.set_property_value('ic/vn-fps')
seanmcleod commented 3 years ago

Yes, I was going to say it looked like you possibly had the aircraft flying backwards with it pointed northwards but ic/vn-fps = -55.

What range of starting configurations are you looking to use for your RL? In particular will the aircraft always start out in a non-stalled condition? If so you need to make sure that the velocities and attitude you start with don't result in the AoA being greater than the critical/stall angle of attack. Also starting with a large AoS (Angle of Sideslip) will also start things off with large forces that have to be corrected if the control positions are all left in their neutral position.

If you do want to include starting conditions that have the aircraft stalled from which the RL agent is expected to recover then you need to make sure that the JSBSim model you're using has valid data past the stall angle, the vast majority of existing JSBSim models don't.

Nx, Ny, Nz are the accelerations in the aircraft's body axes, e.g. think of a set of accelerometers being strapped to the aircraft measuring the accelerations. Given Nx, Ny, Nz and the aircraft's attitude you could transform them to a local NED frame of reference.

I don't see JSBSim exposing local NED accelerations.

  PropertyManager->Tie("accelerations/udot-ft_sec2", this, eU, (PMF)&FGAccelerations::GetUVWdot);
  PropertyManager->Tie("accelerations/vdot-ft_sec2", this, eV, (PMF)&FGAccelerations::GetUVWdot);
  PropertyManager->Tie("accelerations/wdot-ft_sec2", this, eW, (PMF)&FGAccelerations::GetUVWdot);

  /** Retrieves a body frame acceleration component.
      Retrieves a body frame acceleration component. The acceleration returned
      is extracted from the vUVWdot vector (an FGColumnVector3). The vector for
      the acceleration in Body frame is organized (Ax, Ay, Az). The vector is
      1-based. In other words, GetUVWdot(1) returns Ax. Various convenience
      enumerators are defined in FGJSBBase. The relevant enumerators for the
      acceleration returned by this call are, eX=1, eY=2, eZ=3.
      units ft/sec^2
      @param idx the index of the acceleration component desired (1-based).
      @return The body frame acceleration component.
  */
  double GetUVWdot(int idx) const { return vUVWdot(idx); }

  PropertyManager->Tie("accelerations/Nx", this, &FGAuxiliary::GetNx);
  PropertyManager->Tie("accelerations/Ny", this, &FGAuxiliary::GetNy);
  PropertyManager->Tie("accelerations/Nz", this, &FGAuxiliary::GetNz);

  /** The longitudinal acceleration in g's of the aircraft center of gravity. */
  double GetNx            (void) const { return Nx;         }

  /** The lateral acceleration in g's of the aircraft center of gravity. */
  double GetNy            (void) const { return Ny;         }

  /** The vertical acceleration in g's of the aircraft center of gravity. */
  double GetNz            (void) const { return Nz;         }
seanmcleod commented 3 years ago

@JDatPNW what is the aim of the RL agent? Glancing at your current reward function it seems maximum reward is set when the pitch and roll angles are close to 0? So is the idea for the agent to be able to take an aircraft in any starting attitude and have the aircraft get to 0 pitch and roll?

JDatPNW commented 3 years ago

Hi @seanmcleod and @bcoconni. Thanks for the replies! I will change all my set and get calls to vn = self.fdm['ic/vn-fps'] like you suggested. Knowing that the properties I am setting actually exist is really reassuring. Thanks for pointing that out!

As for the -50 starting velocity. That was my fault, and I only noticed it when I typed it up yesterday. XPlane seems to be South oriented (S = + / N = -), and since I was writing the JSBSim Code on the XPlane base, a few of these mistakes slipped their way into my code. I found a few yesterday and corrected them, which means the plane is no longer looping at the beginning.

The reason why I am using ic/vn-fps, ic/ve-fps and ic/vd-fps is because XPLane, to the best of my knowledge, only lets me set these values to set the planes velocity, so sadly I am somewhat bound to them. I want my code to be modular with it's environments, since I would like to see if I can train an agent that can learn it's policy in JSBSim and still function in XPlane for example.

I tried calculating my own local NED accelerations, since I have all values I need for them. I think (hope) that should work, if I did not make any mistakes.

On the topic of stalls. From what I understand a stall occurs if the AoA exceeds the critical value. (So for example if the plane is at a pitch of 90 degrees pointing up, as an extreme example, right?) I do not necessarily need/want any stall angles, but I can also not say that they will not occur throughout training, since the agent performs random actions for a long time and might enter them somewhat regularly at times. When you say most models don't have valid data past the stall point, does that mean that I will not behave physically accurate once it has exceeded the critical angles? (Currently I am using the c172r) As for the 25 and 45 degrees, these I just made up to be honest, and I would be fine changing them. My thought was that spawning the plane in these conditions will expose it to many states and therefore help it with generalization later down the line. But if the angles currently chosen are just unrealistic (here my lack of aviation expertise becomes obvious again. And sadly I also don't really know anyone that does, so more often that I would like to admit I guesstimate some of these values)

Lastly, yes! I am trying to train the agent to stabilize at 0°, 0° - for roll and pitch respectively. And if I manage to do so, potentially expand that to the agent stabilizing at any given pitch and roll values.

Thanks for helping me out with all this! I really appreciate it :) I am trying my best to incorporate JSBSim and to make it interchangeable with XPlane, but my limited knowledge in terms of aviation sometimes makes things a bit more confusing to me than they probably are. And with XPlane and JSBSim having a lot of axis inverted and using different measurement units, I ended up making a good amount of mistakes in the beginning. I hope I have gotten rid of most of them, and I will try to crosscheck them one more time today. Is there any good ways of going about that? I am just booting up both sims, and then going through the properties at flight time (while paused) and write down which axis is pointing which way) Thanks!! JD

seanmcleod commented 3 years ago

@JDatPNW reading through the Markov-Pilot thesis you referenced I saw the following comment about using XPlane and JSBSim for RL.

For RL experiments it is of utter importance to use a fast model that can speed up beyond real-time providing a convenient interface to issue control commands and to yield internal state information.

Eventually, JSBSim was chosen as flight dynamics model. JSBSim can be used in a headless mode for blazingly fast simulations during training

The first experiments in the course of this work were conducted with XPlane, but it became obvious that this simulator is extremely difficult to integrate in the RL-interaction cycle: XPlane runs in real-time with no easy to access possibility to perform discrete steps. It runs in real-time and even slows down the internal simulation time without further notice if the rendering is slow. It is not possible to run XPlane in a headless mode without rendering the scenery, which consumes a lot of GPU power that cannot be used for ANN training anymore. All in all, the experiments with XPlane were quite disappointing as the mentioned constraints made learning slow and debugging almost infeasible.

In terms of your question:

When you say most models don't have valid data past the stall point, does that mean that I will not behave physically accurate once it has exceeded the critical angles?

Correct. Although I would imagine that typically exceeding the critical angle of attack would be a reason to quit the RL episode? Unless you're specifically designing an agent to recover from a stall.

From what I understand a stall occurs if the AoA exceeds the critical value. (So for example if the plane is at a pitch of 90 degrees pointing up, as an extreme example, right?)

A pitch angle of 90 degrees by itself doesn't mean the AoA is greater than the critical angle, the AoA is the angle between the pitch angle and the velocity vector. So it can be exceeded at any pitch angle, and the AoA can be smaller than the critical angle at any pitch angle, e.g. aircraft performing a loop pitch up and past 90 degrees without stalling.

JDatPNW commented 3 years ago

@JDatPNW reading through the Markov-Pilot thesis you referenced I saw the following comment about using XPlane and JSBSim for RL.

For RL experiments it is of utter importance to use a fast model that can speed up beyond real-time providing a convenient interface to issue control commands and to yield internal state information.

Eventually, JSBSim was chosen as flight dynamics model. JSBSim can be used in a headless mode for blazingly fast simulations during training

The first experiments in the course of this work were conducted with XPlane, but it became obvious that this simulator is extremely difficult to integrate in the RL-interaction cycle: XPlane runs in real-time with no easy to access possibility to perform discrete steps. It runs in real-time and even slows down the internal simulation time without further notice if the rendering is slow. It is not possible to run XPlane in a headless mode without rendering the scenery, which consumes a lot of GPU power that cannot be used for ANN training anymore. All in all, the experiments with XPlane were quite disappointing as the mentioned constraints made learning slow and debugging almost infeasible.

I came to the same conclusion, which is why I wanted to implement JSBSim. I would want to test the trained model (which I trained using JSBIM) on XPlane, to see if it can perform in a different environment/if it can generalize. Which is why I need the action space and the state space to be the same. I don't really plan to train using XPlane anymore, because I came to the same conclusion like the one you quoted.

In terms of your question:

When you say most models don't have valid data past the stall point, does that mean that I will not behave physically accurate once it has exceeded the critical angles?

Correct. Although I would imagine that typically exceeding the critical angle of attack would be a reason to quit the RL episode? Unless you're specifically designing an agent to recover from a stall.

That is a good point, that I have honestly not really thought about yet.

From what I understand a stall occurs if the AoA exceeds the critical value. (So for example if the plane is at a pitch of 90 degrees pointing up, as an extreme example, right?)

A pitch angle of 90 degrees by itself doesn't mean the AoA is greater than the critical angle, the AoA is the angle between the pitch angle and the velocity vector. So it can be exceeded at any pitch angle, and the AoA can be smaller than the critical angle at any pitch angle, e.g. aircraft performing a loop pitch up and past 90 degrees without stalling.

Oh ok I see. I misunderstood that then. Do you know if and how I can test if the stall angle has been reached? I do think stopping the simulation and marking it as a failure would make a lot of sense, just like you suggested. When looking at XPlane I found these: XPlane Name Type Writable Unit Desription
sim/aircraft/view/acf_has_stallwarn int y bool has audio stall warning?
  sim/aircraft/overflow/acf_stall_warn_alpha float y ??? alpha of stall warning... user must specify since warning different for different planes.
  sim/flightmodel/failures/stallwarning int y ??? Stall Warning
  sim/operation/failures/rel_stall_warn int y failure_enum Stall warning has failed
  sim/cockpit2/annunciators/stall_warning int n boolean Stall warning going off, yes or no.
  sim/cockpit2/annunciators/stall_warning_ratio float n 0..1 For aircraft with variable stall warnings, 0 = no stall, 1 = full stall
  sim/flightmodel2/wing/elements/element_is_stalled float[320] n boolean Per element - is this element stalled?

For JSBSim I found the following: JSBSim

So it looks like I could check for stalls occurring. I can not test these out right now, I will give them a try next week. I did also not manage to crosscheck all the properties with the XPlane DREFs today, I hope to get that done next week as well. I did however start a job with the JSBSim model, trying to train it (with new starting angles. I believe I used 10 and 20, instead of the 25 and 45).

Thanks a ton! JD