Fix calculated values according to ISO 2533:1975 table

The isa.rb file implements the formulas provided in ISO 2533:1975. The goal is to be able to generate the tables through code instead of a lookup in the document.

This document is identical to ICAO Doc 7488/3, which is provided publicly. The only differences are variable names, where in ICAO 7488 they use a suffix notation of _0 ("p_0" for pressure at mean sea level), in ISO 2533 the suffix _n ("p_n") is used.

Here we provide the formula definitions in the document, and also a few pages of the sample text.

Currently when you run isa.rb, it performs a calculation using code and also compares against the hard coded numbers.

There are a few things you may want to do to better compare:

round up numbers to their expected comparison granularity
add more tests in accordance to the tables

The calculated values for a number of formulas are wrong, especially for those that depend on the "pressure" formula which does not produce correct results.

The implementation of the following formulas need to be fixed:

Clause 2.7:
- Pressure: Formula (12) and (13)
Clause 2.8
- Density: Formula (14)
- Specific weight: Formula (15)
Clause 2.9
- Pressure scale height: Formula (16)
Clause 2.10
- Air number density: Formula (17)
Clause 2.12
- Mean free path of air particles: Formula (19)
Clause 2.13
- Air-particle collision frequency: Formula (20)
Clause 2.15
- Dynamic viscosity: Formula (22)
Clause 2.17
- Thermal conductivity: Formula (24)

Sample output of tests

TEST CASE 1: (calculated vs target)
  H =   -2000.0     (-2000.0)
  h =   -1999.370947130308      (-1999.0)
  TK    =   301.15      (301.15)
  TC    =   28.0        (28.0)
  p_mbar    =   1013.2500857056044      (1277.74)
  p_mmhg    =   760.0003857845171       (958.382)
  rho   =   1.1721194748699297      (1.47808)
  g =   9.812821812969393       (9.8128)
  p_p_n =   1.0000000845848551      (1.26103)
  rho_rho_n =   0.956832224383616       (1.20659)
  root_rho_rho_n    =   0.9781780126253177      (1.09845)
  a =   347.88555921638397      (347.886)
  mu    =   1.851438196178237e-05       (1.8514e-05)
  v =   1.5113781193291728e-05      (1.2526e-05)
  lambda    =   0.024274052858840104        (0.026359)
  H_p   =   8815.036029581692       (8809.5)
  gamma =   11.501799550409876      (14.504)
  n =   2.437187278692176e+28       (3.0734e+25)
  v_bar =   469.18315108853415      (469.18)
  omega =   7123251097564.441       (8535100000.0)
  l =   6767378755316.568       (549710000.0)

TEST CASE 2: (calculated vs target)
  H =   -1900.0     (-1900.0)
  h =   -1899.4322708568118     (-1899.0)
  TK    =   300.5       (300.5)
  TC    =   27.350000000000023      (27.35)
  p_mbar    =   1013.2500654526405      (1263.31)
  p_mmhg    =   760.0003705935384       (947.559)
  rho   =   1.1746548179753653      (1.46455)
  g =   9.812513176210215       (9.8125)
  p_p_n =   1.0000000645967337      (1.24679)
  rho_rho_n =   0.9589018922247879      (1.19555)
  root_rho_rho_n    =   0.9792353609959089      (1.09341)
  a =   347.50991956450883      (347.51)
  mu    =   1.848366536226033e-05       (-3.1516)
  v =   1.5088706418171697e-05      (-3.7379)
  lambda    =   0.024220851683704397        (0.026308)
  H_p   =   8796.009719041336       (8709.8)
  gamma =   11.526315878882084      (14.371)
  n =   2.4424590159989667e+28      (3.0432e+25)
  v_bar =   468.67653679865793      (468.68)
  omega =   7130950808795.067       (8447800000.0)
  l =   6774693794684.201       (554790000.0)

Definition of formulas

Values

metanorma / atmospheric