BRDF All-NaN error (hard to reproduce)

[oceancolorcoder]

While batch-reprocessing pySAS data beginning with FICE22, I encountered an odd new error in FRM processing.

In the Full-FRM regime I have successfully batched the entire FICE22 cruise (with and without station extraction) for M99MA, M99NN, M99SS, Z17MA, and Z17SS. While running the last configuration, Z17NN, it runs up to …_20220718_090000… with no errors, but the next file, …_20220718_1200000… fails with the error:

FRM Processing: ES
100 of the 100 MC samples were not processed correctly (contained only nans) and will be ignored in the punpy output
FRM Processing: LI
FRM Processing: LT
100 of the 100 MC samples were not processed correctly (contained only nans) and will be ignored in the punpy output
100 of the 100 MC samples were not processed correctly (contained only nans) and will be ignored in the punpy output
100 of the 100 MC samples were not processed correctly (contained only nans) and will be ignored in the punpy output
Uncertainty Update Elapsed Time: 40.4 s
Processing MODISA
Filtering reflectance spectra for negative values.
0.0% of Rrs_HYPER spectra flagged
0.0% of nLw_HYPER spectra flagged
Processing chlor_a
Processing Wei QA
/Users/daurin/GitRepos/HyperCP/Source/L2wei_QA.py:264: RuntimeWarning: Mean of empty slice
  totScore = np.nanmean(C, 0)
Processing avw
Processing QWIP
Processing CDOM, Sg, DOC
Processing qaa
Applying iterative Morel et al. 2002 BRDF correction to Rrs and nLw
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:562: RuntimeWarning: All-NaN slice encountered
  minval = np.nanmin(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:563: RuntimeWarning: All-NaN slice encountered
  maxval = np.nanmax(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:562: RuntimeWarning: All-NaN slice encountered
  minval = np.nanmin(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:563: RuntimeWarning: All-NaN slice encountered
  maxval = np.nanmax(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:562: RuntimeWarning: All-NaN slice encountered
  minval = np.nanmin(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:563: RuntimeWarning: All-NaN slice encountered
  maxval = np.nanmax(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:562: RuntimeWarning: All-NaN slice encountered
  minval = np.nanmin(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:563: RuntimeWarning: All-NaN slice encountered
  maxval = np.nanmax(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:562: RuntimeWarning: All-NaN slice encountered
  minval = np.nanmin(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:563: RuntimeWarning: All-NaN slice encountered
  maxval = np.nanmax(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:562: RuntimeWarning: All-NaN slice encountered
  minval = np.nanmin(new_x.values)
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/xarray/core/missing.py:563: RuntimeWarning: All-NaN slice encountered
  maxval = np.nanmax(new_x.values)
Traceback (most recent call last):
  File "/Users/daurin/GitRepos/HyperCP/Main.py", line 492, in singleL2Clicked
    self.processSingle('L2')
  File "/Users/daurin/GitRepos/HyperCP/Main.py", line 464, in processSingle
    Controller.processFilesSingleLevel(self.outputDirectory,fileNames, calibrationMap, level, flag_Trios)
  File "/Users/daurin/GitRepos/HyperCP/Source/Controller.py", line 870, in processFilesSingleLevel
    Controller.processSingleLevel(pathOut, fp, calibrationMap, level, flag_Trios)
  File "/Users/daurin/GitRepos/HyperCP/Source/Controller.py", line 736, in processSingleLevel
    root = Controller.processL2(root,outFilePath)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/Controller.py", line 449, in processL2
    node = ProcessL2.processL2(root,station)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/ProcessL2.py", line 2334, in processL2
    ProcessL2BRDF.procBRDF(node, BRDF_option='M02')
  File "/Users/daurin/GitRepos/HyperCP/Source/ProcessL2BRDF.py", line 128, in procBRDF
    arr[mask] = np.interp(np.flatnonzero(mask), np.flatnonzero(~mask), arr[~mask])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<__array_function__ internals>", line 180, in interp
  File "/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/numpy/lib/function_base.py", line 1594, in interp
    return interp_func(x, xp, fp, left, right)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: array of sample points is empty

[oceancolorcoder]

I can pick up processing after that file, but encounter the same error again at file ...20220719_100000...

[AlexisDeru]

I think this is not linked to the BRDF code, but more to Py6S. This is cases where py6s model are all NaN. Is this an error of py6s or is it specific cases where the py6s model is not applicable, I am not sure yet ?

[oceancolorcoder]

The problem in the PIU.py MC run is in HyperOCR.FRM at L2016 calculating sample_data5. Clearly those all-NaNs cascade down to produce errors in the rest of L2, including BRDF (where it crashes).

[oceancolorcoder]

The problem does stem from failed 6S model at L1B resulting in all NaNs in the model AND in Es. Also, the crashing at BRDF was fixed with a recent commit from JIG. Closing this issue and opening a new one.