isofit / isofit

Imaging Spectrometer Optimal FITting (ISOFIT) contains a set of routines and utilities for fitting surface, atmosphere and instrument models to imaging spectrometer data.
Apache License 2.0
86 stars 82 forks source link

Failing dev-branch isofit runs on the cluster with small images #569

Closed evan-greenbrg closed 1 week ago

evan-greenbrg commented 1 week ago

Apologies if this isn't the place for this type of debugging issue. I'm having some difficulty running dev-branch isofit with modtran on the EMIT cluster. I'm seeing a fairly consistent KeyError.

For example:

  File "/home/bgreenbe/Github/isofit/isofit/radiative_transfer/engines/modtran.py", line 310, in makeSim
    if vals["OBSZEN"] < 90:
       ~~~~^^^^^^^^^^
KeyError: 'OBSZEN'

The caveat is that I'm running small test images. In the extreme case below, it's a 25x25 pixel image. This means that the geometric lut grid elements are generally smaller than the specified grid spacing, and should be passed through as constant dimensions.

I've spent this morning and yesterday afternoon trying to debug the issue and running the pipeline in multiple ways. Initially I though the issue was caused by me manually editing config files. However, the run configs pass all checks. The motivation for posting here is that the baseline dev-branch apply_oe cli kick off fails in the same manner. I'm attaching my failing .sh script below:

#! /bin/sh                                                              
rdn="/beegfs/store/bgreenbe/emit/small_ang20230723t170821_sample/ang20230723t170821_rdn"
loc="/beegfs/store/bgreenbe/emit/small_ang20230723t170821_sample/ang20230723t170821_loc"
obs="/beegfs/store/bgreenbe/emit/small_ang20230723t170821_sample/ang20230723t170821_obs"
root="/beegfs/store/bgreenbe/emit/small_ang20230723t170821_sample"
sensor="ang'                                                            
surface_path="/home/bgreenbe/Multisurface/surface_SWIPE_avng_swir_20240424.mat"
modtran_path=$MODTRAN_DIR                                               
presolve="true"                                                         
n_cores=40                                                              
logfile="log.txt"

isofit apply_oe $rdn $loc $obs $root $sensor --surface_path $surface_path --modtran_path $modtran_path --presolv
    e --n_cores $n_cores --log_file $logfile

If anyone has the time, could someone confirm that they are able to reproduce the issue, or even better, to successfully initiate the LUT generation with dev-branch isofit and modtran on the cluster? I'm trying to nail down if this is a configuration issue on my end or not.

evan-greenbrg commented 1 week ago

Same issue as mentioned in #571