metanorma / atmospheric

International Standard Atmosphere / ICAO Standard Atmosphere (ISA) from ICAO 7488 / ISO 2533
BSD 2-Clause "Simplified" License
1 stars 1 forks source link

Improve accuracy, implement rounding #5

Closed newbthenewbd closed 7 months ago

newbthenewbd commented 7 months ago

Following these changes, the test run looks like this:

PRE-CALCULATED PRESSURE LAYERS:
[127773.72968448336,
 101325.0,
 22632.040596934745,
 5474.877660660028,
 868.0158377493663,
 110.90578455391473,
 66.93853537303916,
 3.956392754582872,
 0.8862723913002989]

TEST CASE 1: (calculated vs target)
  H =   -2000.0     (-2000.0)
  h =   -1999       (-1999.0)
  TK    =   301.15      (301.15)
  TC    =   28.0        (28.0)
  p_mbar    =   1277.74     (1277.74)
  p_mmhg    =   958.382     (958.382)
  rho   =   1.47808     (1.47808)
  g =   9.8128      (9.8128)
  p_p_n =   1.26103     (1.26103)
  rho_rho_n =   1.20659     (1.20659)
  root_rho_rho_n    =   1.09845     (1.09845)
  a =   347.886     (347.886)
  mu    =   1.8514e-05      (1.8514e-05)
  v =   1.2526e-05      (1.2526e-05)
  lambda    =   0.026359        (0.026359)
  H_p   =   8809.5      (8809.5)
  gamma =   14.504      (14.504)
  n =   30734000000000000000000000      (3.0734e+25)
  v_bar =   469.18      (469.18)
  omega =   8535100000      (8535100000.0)
  l =   5.4971e-08      (5.4971e-08)

TEST CASE 2: (calculated vs target)
  H =   -1900.0     (-1900.0)
  h =   -1899       (-1899.0)
  TK    =   300.5       (300.5)
  TC    =   27.35       (27.35)
  p_mbar    =   1263.31     (1263.31)
  p_mmhg    =   947.559     (947.559)
  rho   =   1.46455     (1.46455)
  g =   9.8125      (9.8125)
  p_p_n =   1.24679     (1.24679)
  rho_rho_n =   1.19555     (1.19555)
  root_rho_rho_n    =   1.09341     (1.09341)
  a =   347.51      (347.51)
  mu    =   1.8484e-05      (1.8484e-05)
  v =   1.2621e-05      (1.2621e-05)
  lambda    =   0.026308        (0.026308)
  H_p   =   8790.8      (8790.8)
  gamma =   14.371      (14.371)
  n =   30452000000000000000000000      (3.0452e+25)
  v_bar =   468.68      (468.68)
  omega =   8447800000      (8447800000.0)
  l =   5.5479e-08      (5.5479e-08)

Metanorma PR checklist

newbthenewbd commented 7 months ago

3

ronaldtse commented 7 months ago

@newbthenewbd in a message if you write "Fixes #3" then when this PR merges, the issue will automatically get closed.

ronaldtse commented 7 months ago

Fixes #3.

newbthenewbd commented 7 months ago

@newbthenewbd in a message if you write "Fixes #3" then when this PR merges, the issue will automatically get closed.

I realize, but didn't know whether you wanted this haha :) Thanks!