nasa / HyperCP

Other
39 stars 22 forks source link

PIU returns random=None for Lw_Convolved #216

Closed oceancolorcoder closed 1 month ago

oceancolorcoder commented 2 months ago

KORUS file listed below (and placed on the Box for testing) crashes when PIU.py returns "random" as None to Uncertainty_Analysis.Propagate_Lw_Convolved L209. Processing is Factory, M99MA.

Process Single Level
ProcessL2: /Users/daurin/Projects/HyperPACE/field_data/HyperSAS/KORUS/L1BQC/1614504_L1BQC.hdf
stationsEnsemblesReflectance
**************Cannot do cubic spline interpolation, length of datasets < 3
Binning datasets to ensemble time interval.
Convolving MODIS Aqua (ir)radiances in the slice
1 spectra in slice (ensemble).
Calculating M99 glint correction with complete LUT
Calculating Zhang glint correction.
Zhang17 Elapsed Time: 3.3 s
Reading : Data/hybrid_reference_spectrum_p1nm_resolution_c2020-09-21_with_unc.nc
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/comet_maths/linear_algebra/matrix_calculation.py:274: UserWarning: One of the correlation matrices is not positive definite. It has been slightly changed (maximum difference of 6.661338147750939e-16) to accomodate our method.
  warnings.warn(
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/comet_maths/linear_algebra/matrix_calculation.py:274: UserWarning: One of the correlation matrices is not positive definite. It has been slightly changed (maximum difference of 4.440892098500626e-16) to accomodate our method.
  warnings.warn(
/opt/anaconda3/envs/hypercp/lib/python3.11/site-packages/comet_maths/linear_algebra/matrix_calculation.py:274: UserWarning: One of the correlation matrices is not positive definite. It has been slightly changed (maximum difference of 6.661338147750939e-16) to accomodate our method.
  warnings.warn(
Backend QtAgg is interactive backend. Turning interactive mode on.
Traceback (most recent call last):
  File "/Users/daurin/GitRepos/HyperCP/Main.py", line 492, in singleL2Clicked
    self.processSingle('L2')
  File "/Users/daurin/GitRepos/HyperCP/Main.py", line 464, in processSingle
    Controller.processFilesSingleLevel(self.outputDirectory,fileNames, calibrationMap, level, flag_Trios)
  File "/Users/daurin/GitRepos/HyperCP/Source/Controller.py", line 870, in processFilesSingleLevel
    Controller.processSingleLevel(pathOut, fp, calibrationMap, level, flag_Trios)
  File "/Users/daurin/GitRepos/HyperCP/Source/Controller.py", line 736, in processSingleLevel
    root = Controller.processL2(root,outFilePath)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/Controller.py", line 449, in processL2
    node = ProcessL2.processL2(root,station)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/ProcessL2.py", line 2302, in processL2
    if not ProcessL2.stationsEnsemblesReflectance(node, root,station):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/ProcessL2.py", line 2193, in stationsEnsemblesReflectance
    if not ProcessL2.ensemblesReflectance(node, sasGroup, referenceGroup, ancGroup, uncGroup, esRawGroup,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/ProcessL2.py", line 1748, in ensemblesReflectance
    xUNC.update(instrument.rrsHyperUNCFACTORY(node, uncGroup, rhoScalar, rhoVec, rhoUNC, waveSubset, xSlice))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/ProcessInstrumentUncertainties.py", line 1215, in rrsHyperUNCFACTORY
    output["lwUNC_MODISA"] = Convolve.Propagate_Lw_Convolved(lw_means, lw_uncertainties,
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daurin/GitRepos/HyperCP/Source/Uncertainty_Analysis.py", line 230, in Propagate_Lw_Convolved
    return np.sqrt(random ** 2 + systematic ** 2)
                   ~~~~~~~^^~~
TypeError: unsupported operand type(s) for ** or pow(): 'NoneType' and 'int'
oceancolorcoder commented 2 months ago

Also for file 1614701_L1BQC.hdf (copied to Box)

oceancolorcoder commented 1 month ago

Resolved with 0454f9135bd4f3ba316eb2768fefc68cfc05c96a