Bad Wavelengths Return NaN Irradiance, e.g. 761 nm

[csarantos-ceres]

Certain wavelengths such as 761 nm are returning NaN irradiance (in some cases over a 1.2 nm range corresponding to two sampling points at the 10 cm^-1 resolution?). I am putting in 30% broadband surface albedo and moderate AOT and water content, have not experimented with reducing those yet to see if this still occurs.

I found this 2014 thread here that implies this may be a 6S issue. I'm not sure if there is more up to date information or a fix for this, or if this is still a fundamental 6S issue?

https://groups.google.com/forum/m/#!topic/py6s/vUwoQ_DWBTk

Wavelengths at which I'm getting NaN irradiance (grid with 1 nm spacing):

Full debug report and output full text

Py6S Debugging Report
---------------------
Run on 2020-06-12 11:04:05.180328
Platform: Windows-10-10.0.18362-SP0
Python version: 3.7.6 | packaged by conda-forge | (default, Jun  1 2020, 18:11:50) [MSC v.1916 64 bit (AMD64)]
Py6S version: 1.8.0
---------------------
6S wrapper script by Robin Wilson
Using 6S located at C:\Users\Ceres\AppData\Local\conda\conda\envs\improc\Library\bin\sixs.exe
Running 6S using a set of test parameters
6sV version: 1.1
The results are:
Expected result: 619.158000
Actual result: 619.158000
#### Results agree, Py6S is working correctly
---------------------
0 (User defined)
21.307878 133.471212 0.000000 0.000000 6 5
8 (Water Vapour and Ozone)
2.981510 0.324494
1
0
0.093113 value
0.000000
0.000000
-1
0.500000
0 Homogeneous surface
0 No directional effects
0
0.3
-1 No atm. corrections selected

Output full text:

******************************* 6SV version 1.1 *******************************
*                                                                             *
*                       geometrical conditions identity                       *
*                       -------------------------------                       *
*                       user defined conditions                               *
*                                                                             *
*   month:  6 day :   5                                                       *
*   solar zenith angle:   21.31 deg  solar azimuthal angle:      133.47 deg   *
*   view zenith angle:     0.00 deg  view azimuthal angle:         0.00 deg   *
*   scattering angle:    158.69 deg  azimuthal angle difference: 133.47 deg   *
*                                                                             *
*                       atmospheric model description                         *
*                       -----------------------------                         *
*           atmospheric model identity :                                      *
*             user defined water content : uh2o= 2.982 g/cm2                  *
*             user defined ozone content : uo3 = 0.324 cm-atm                 *
*           aerosols type identity :                                          *
*               Continental aerosol model                                     *
*           optical condition identity :                                      *
*               visibility : 69.61 km  opt. thick. 550 nm :  0.0931           *
*                                                                             *
*                       spectral condition                                    *
*                       ------------------                                    *
*            monochromatic calculation at wl 0.761 micron                     *
*                                                                             *
*                       Surface polarization parameters                       *
*                       ----------------------------------                    *
*                                                                             *
*                                                                             *
* Surface Polarization Q,U,Rop,Chi    0.00000  0.00000  0.00000     0.00      *
*                                                                             *
*                                                                             *
*                       target type                                           *
*                       -----------                                           *
*           homogeneous ground                                                *
*             monochromatic reflectance  0.300                                *
*                                                                             *
*                       target elevation description                          *
*                       ----------------------------                          *
*           ground pressure  [mb] 1013.00                                     *
*           ground altitude  [km] 0.000                                       *
*                                                                             *
*                       plane simulation description                          *
*                       ----------------------------                          *
*           plane  pressure          [mb] 1013.00                             *
*           plane  altitude absolute [km]  0.000                              *
*                atmosphere under plane description:                          *
*                ozone content             0.000                              *
*                h2o   content             0.000                              *
*               aerosol opt. thick. 550nm  0.000                              *
*                                                                             *
*******************************************************************************

*******************************************************************************
*                                                                             *
*                         integrated values of  :                             *
*                         --------------------                                *
*                                                                             *
*       apparent reflectance        NaN  appar. rad.(w/m2/sr/mic)      NaN    *
*                   total gaseous transmittance  0.378                        *
*                                                                             *
*******************************************************************************
*                                                                             *
*                         coupling aerosol -wv  :                             *
*                         --------------------                                *
*           wv above aerosol :     NaN     wv mixed with aerosol :     NaN    *
*                       wv under aerosol :     NaN                            *
*******************************************************************************
*                                                                             *
*                         integrated values of  :                             *
*                         --------------------                                *
*                                                                             *
*       app. polarized refl.  0.0000    app. pol. rad. (w/m2/sr/mic)    0.000 *
*             direction of the plane of polarization  0.00                    *
*                   total polarization ratio       NaN                        *
*                                                                             *
*******************************************************************************
*                                                                             *
*                         int. normalized  values  of  :                      *
*                         ---------------------------                         *
*                      % of irradiance at ground level                        *
*     % of direct  irr.    % of diffuse irr.    % of enviro. irr              *
*               0.000               0.000               0.000                 *
*                       reflectance at satellite level                        *
*     atm. intrin. ref.   background  ref.  pixel  reflectance                *
*               0.000                 NaN                 NaN                 *
*                                                                             *
*                         int. absolute values of                             *
*                         -----------------------                             *
*                      irr. at ground level (w/m2/mic)                        *
*     direct solar irr.    atm. diffuse irr.    environment  irr              *
*                 NaN                 NaN                 NaN                 *
*                      rad at satel. level (w/m2/sr/mic)                      *
*     atm. intrin. rad.    background  rad.    pixel  radiance                *
*               0.000                 NaN                 NaN                 *
*                                                                             *
*                                                                             *
*                      sol. spect (in w/m2/mic)                               *
*                                1207.242                                     *
*                                                                             *
*******************************************************************************

*******************************************************************************
*                                                                             *
*                          integrated values of  :                            *
*                          --------------------                               *
*                                                                             *
*                             downward        upward          total           *
*      global gas. trans. :     0.37803        1.00000        0.37803         *
*      water   "     "    :     1.00000        1.00000        1.00000         *
*      ozone   "     "    :     0.99748        1.00000        0.99748         *
*      co2     "     "    :     1.00000        1.00000        1.00000         *
*      oxyg    "     "    :     0.37899        1.00000        0.37899         *
*      no2     "     "    :     1.00000        1.00000        1.00000         *
*      ch4     "     "    :     1.00000        1.00000        1.00000         *
*      co      "     "    :     1.00000        1.00000        1.00000         *
*                                                                             *
*                                                                             *
*      rayl.  sca. trans. :         NaN            NaN            NaN         *
*      aeros. sca.   "    :         NaN            NaN            NaN         *
*      total  sca.   "    :         NaN            NaN            NaN         *
*                                                                             *
*                                                                             *
*                                                                             *
*                             rayleigh       aerosols         total           *
*                                                                             *
*      spherical albedo   :         NaN            NaN            NaN         *
*      optical depth total:         NaN        0.06524            NaN         *
*      optical depth plane:     0.00000        0.00000        0.00000         *
*      reflectance I      :     0.00000        0.00000        0.00000         *
*      reflectance Q      :     0.00000        0.00000        0.00000         *
*      reflectance U      :     0.00000        0.00000        0.00000         *
*      polarized reflect. :     0.00000        0.00000        0.00000         *
*      degree of polar.   :         NaN           0.00            NaN         *
*      dir. plane polar.  :      -45.00         -45.00         -45.00         *
*      phase function I   :     1.38442        0.23668            NaN         *
*      phase function Q   :    -0.09495        0.04385            NaN         *
*      phase function U   :    -1.33978       -0.13987            NaN         *
*      primary deg. of pol:    -0.06858        0.18526            NaN         *
*      sing. scat. albedo :     1.00000        0.87229            NaN         *
*                                                                             *
*                                                                             *
*******************************************************************************

[csarantos-ceres]

It looks like maybe gaseous transmission is correclty calculated but aerosol coupling is NaN? Not sure, though.

[robintw]

Hi @csarantos-ceres.

Unfortunately I don't have any further information on this.

It's definitely an issue with the underlying 6S model, as it occurs when 6S is run outside of Py6S too. It also seems to be platform related, as it doesn't occur on my OS X machine, but occurs for you and the person in the original forum message, who were both using Windows. I think it must be something related to the compilation of the 6S code on Windows and some sort of floating point error - but beyond that I can't really say.

I don't have access to a Windows machine at the moment for testing - but is there any chance you could run this code on a different platform?

I've contacted the original author of 6S about this before, but haven't had any luck with a response. Feel free to contact him yourself (he's Eric Vermote) and do let me know if you hear anything.

[csarantos-ceres]

Thanks, Robin! That's still very helpful to know that it doesn't happen in OSX. I'm ultimately intending to run this in Linux and this work is just an intermediate step. So if the Linux versions don't have this issue, that would be excellent! If I end up running it in Linux, I'll report back what happens.

Thanks! Chris Sarantos

[csarantos-ceres]

Whoops, I realize that I never reported back what happened. My recollection is that I ran it natively in Linux and this issue went away.

[simonrp84]

@csarantos-ceres @robintw: I spotted this problem today and it is indeed coming from 6S itself. Having looked through the source code, the problem is in interp.f. If you change line 66 of that file from: if(wl.gt.wldis(ll).and.wl.le.wldis(ll+1)) linf=ll To: if(wl.ge.wldis(ll).and.wl.lt.wldis(ll+1)) linf=ll Then the code will output good results (no NaNs, output data looks OK). I've let Eric know via email.

[csarantos-ceres]

Good find! Thanks for notifying Eric.

[josselin-aval]

Hello @robintw and @simonrp84 ,

Do you know if this issue has been corrected in 6S?

In addition if I well understand, Py6S is based on 6SV1.1. However the last version of 6S seems to be the 2.1 (https://salsa.umd.edu/6spage.html). Maybe this bug is not in 6SV2.1. Do you plan to update Py6S with 6S2.1?

Thank you in advance for your support.

[simonrp84]

No, to my knowledge this has not been corrected. I emailed several times and never got a response from any of the team. Perhaps you could drop them a mail in case mine are getting caught by their junk filter?

[josselin-aval]

Hi @simonrp84 , thanks a lot for your response. Good idea, I will send a mail and keep you informed.

[robintw]

I also haven't heard anything from the 6S team, despite contacting them multiple times.

There may be a fix for this in version 2.1, but I haven't yet been able to update Py6S to work with the new version. I now work as a freelancer, and would really need some proper funding to be able to do the upgrade work (which is more complex than it first seems).

[josselin-aval]

Hi @robintw and thanks a lot for your response. I understand your situation.

As I said I will send a mail and keep you informed.

[simonrp84]

@robintw Out of interest (and somewhat offtopic), what's the blocker with upgrading to support V2.1? I simply changed 1.1 to 2.1 in the Py6S code and the results are as expected across all the tests I tried. I presume there's some kind of niche example where it doesn't work?