USF-IMARS / wv-land-cover

:earth_americas: Processing scripts for decision-tree land use classification on worldview 2 imagery
5 stars 5 forks source link

IndexError on `E_glint_y_int[5]` #18

Open 7yl4r opened 5 years ago

7yl4r commented 5 years ago

Seeing the following error:

[airflow@imars-airflow17 tmp-tylar]$ rrs_SQL="area_id=11 AND product_id=37 AND date_time='2018-04-14T16:29:32.987650+00:00' AND provenance='af-ntftorrs_v2'"
[airflow@imars-airflow17 tmp-tylar]$ RRS_SQL="area_id=11 AND product_id=38 AND date_time='2018-04-14T16:29:32.987650+00:00' AND provenance='af-ntftorrs_v2'"
[airflow@imars-airflow17 tmp-tylar]$ MAP_SQL="area_id=11 AND product_id=40 AND date_time='2018-04-14T16:29:32.987650+00:00' AND provenance='af-ntftorrs_v2'"
[...]
[airflow@imars-airflow17 tmp-tylar]$     python3.6 -m wv_classify.wv_classify \
>         $ORTHO_FILE $MET2 $RRS_OUT FAKELOC "EPSG:4326" 2 1
reading geotiff 'ortho/wv02_19890607101112_fake0catalog0id0_u16ns4326.tif'
read 8 bands at resolution 7286x8845
    input size: (7286, 8845, 8)
[...]
 === clearing invalid pixels...
calc mask
reduce  (7286, 8845, 8)
index  (7286, 8845)
1024223 invalid pixels found at x,y:
    (array([   0,    0,    0, ..., 7285, 7285, 7285]), array([   0,    1,    2, ..., 8842, 8843, 8844]))
percent of good pixels in image: 98.41%
 === calculating Rrs...
      Rrs size: (7286, 8845, 8)
writing 7286x8845 '<class 'numpy.float32'>', 8-band geotiff to 'output/wv02_1989060710111_FAKELOC_Rrs.tif'
output/wv02_1989060710111_FAKELOC_Rrs.tif written.
Running DT and/or rrs conversion...
% good pixels by nan-count: 0.0078
n_water 62493384
n_glinted 23687346
807756 px removed with band 0 == 0...
11 px removed w/ band 6 < 0
9 px removed w/ band 7 < 0
62493364 px remain
Deglinting
/opt/wv2_processing/wv_classify/run_rrs.py:228: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  water[:, correction_ind]
least-squares glint correction:
    slope:[1.0790026311946248, 0.7713194766085352, 0.7186440932282508, 0.6358122057187158, 0.8575120607379343, 0.6693314856304513]
    y-int:[0.0009345966483145893, 0.0009616439186358245, -0.0003960467057525194, 0.0023638730344084395, 0.001044694480887087, 0.009017517305385667]
Calculating target class metrics...
/usr/local/lib64/python3.6/site-packages/numpy/core/fromnumeric.py:3257: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
/usr/local/lib64/python3.6/site-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
Executing Deglinting rrs, Bathymetery, and Decision Tree...
deglint:row 1
    1.0790026311946248x0.05693774297833443 - 0.0009345966483145893
    0.7713194766085352x0.07830850034952164 - 0.0009616439186358245
    0.7186440932282508x0.07830850034952164 - -0.0003960467057525194
    0.6358122057187158x0.05693774297833443 - 0.0023638730344084395
    0.8575120607379343x0.07830850034952164 - 0.001044694480887087
    0.6693314856304513x0.05693774297833443 - 0.009017517305385667
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/wv2_processing/wv_classify/wv_classify.py", line 895, in <module>
    main(*sys.argv[1:])
  File "/opt/wv2_processing/wv_classify/wv_classify.py", line 846, in main
    int(dt_out), int(rrs_out)
  File "/opt/wv2_processing/wv_classify/wv_classify.py", line 654, in process_file
    float(E_glint_y_int[5])
IndexError: index 5 is out of bounds for axis 0 with size 5

I can't figure this out because E_glint_y_int is printed above as the y-int array, and it clearly has 6 elements.

Maybe it has something to do with the suspiciously-low-valued "% good pixels by nan-count: 0.0078".