djhoese
commented
11 years ago Hi Kathy, I think I'll need some more information to figure this one out. It looks like the navigation is confusing the python ll2cr. If he could give me more of the output from the ll2cr steps that would be great, the whole log file would be best actually. I'm curious to discover what polar2grid thinks the bounding box of the data is. I say this because from his output you can see that it's flipped from what I would expect:
DEBUG : Real upper-left corner (-1.302259,38.436245) DEBUG : Real lower-right corner (-35.577479,28.536811)
This message is saying that the grid calculated has a west-most longitude of -1 and a east-most longitude of -35...which is not right. The fornav error he was getting about "On next call to fornav, use:" with -1 values usually means the data doesn't lie in the grid. This obviously shouldn't happen when using a dynamic grid like eqc_fit.
If I could get any of the following information it would help a lot:
- The log file from one or more of the failed cases. I'm particularly interested in the ll2cr log messages in those files.
- The grid configuration line for his custom grid that also failed
- Does it fail using the wgs84_fit grid? If it doesn't, it could be a math error on my part.
- Is it possible for him to run a case that doesn't cross the prime meridian? Does that work with the eqc_fit grid?
- (optional) The data for these test cases
Bottom line, we need more EU test cases for everything.
kathys
Dave, we can always ask him for more information if we need to. Kathy
From: Andrew Brooks arb@sat.dundee.ac.uk Date: July 19, 2013 10:36:33 AM CDT Subject: Re: CSPP/IMAPP VIIRS and MODIS polar2grid v1.1 beta reprojection software now available To: Kathy Strabala kathys@ssec.wisc.edu
Hi Kathy
I'm just testing the truecolor part and have stumbled on a problem.
This command seems to work ok: crefl2gtiff.sh -vv --debug --grid-configs $GRIDSCONF -f /data/viirs1/tmp/201307191204/CREFL[IM]*.hdf
This command with a custom grid fails: crefl2gtiff.sh -vv --debug --grid-configs $GRIDSCONF -g msg_2017_ql -f /data/viirs1/tmp/201307191204/CREFL[IM]*.hdf
The msg_2017_ql grid is properly defined and used elsewhere, I should add. It also fails with the built-in ones, eqc_fit for example.
The error is: ERROR : Couldn't find fornav output file 'result_m_nav_crefl_01_msg_2017_ql.real4.2017.2017'
In debug mode it seems to be calling: fornav 3 -v -t f4 f4 f4 -f -999.0 -999.0 -999.0 -F -999.0 -999.0 -999.0 -D 40 -d 2 -s 0 0 3200 48 16 ll2cr_m_nav_msg_2017_ql_cols.real4.3200.768 ll2cr_m_nav_msg_2017_ql_rows.real4.3200.768 image_m_nav_crefl_01.real4.3200.768 image_m_nav_crefl_04.real4.3200.768 image_m_nav_crefl_03.real4.3200.768 2017 2017 result_m_nav_crefl_01_msg_2017_ql.real4.2017.2017 result_m_nav_crefl_04_msg_2017_ql.real4.2017.2017 result_m_nav_crefl_03_msg_2017_ql.real4.2017.2017
which reports some statistics followed by: fill count[0]: 4068289 fill count[1]: 4068289 fill count[2]: 4068289 On next call to fornav, use: chan_scan_first: -1 colrow_scan_first: -1 swath_scans: 0 Removing file: result_m_nav_crefl_01_msg_2017_ql.real4.2017.2017 Removing file: result_m_nav_crefl_04_msg_2017_ql.real4.2017.2017 Removing file: result_m_nav_crefl_03_msg_2017_ql.real4.2017.2017
I don't understand why fornav is failing silently.
Thanks,
Andrew
P.S. Here is a full output from an eqc_fit grid; fornav works for the 3 bands then fails for the single band:
DEBUG : Running fornav with 'fornav 3 -v -t f4 f4 f4 -f -999.0 -999.0 -999.0 -F -999.0 -999.0 -999.0 -D 40 -d 2 -s 0 0 3200 48 16 ll2cr_m_nav_eqc_fit_cols.real4.3200.768 ll2cr_m_nav_eqc_fit_rows.real4.3200.768 image_m_nav_crefl_01.real4.3200.768 image_m_nav_crefl_04.real4.3200.768 image_m_nav_crefl_03.real4.3200.768 15262.0 4408.0 result_m_nav_crefl_01_eqc_fit.real4.15262.4408 result_m_nav_crefl_04_eqc_fit.real4.15262.4408 result_m_nav_crefl_03_eqc_fit.real4.15262.4408' fornav: $Header: /disks/megadune/data/tharan/ms2gth/src/fornav/fornav.c,v 1.32 2012/07/24 21:42:17 tharan Exp $ chan_count = 3 swath_cols = 3200 swath_scans = 48 swath_rows_per_scan = 16 swath_col_file = ll2cr_m_nav_eqc_fit_cols.real4.3200.768 swath_row_file = ll2cr_m_nav_eqc_fit_rows.real4.3200.768 swath_chan_file[0] = image_m_nav_crefl_01.real4.3200.768 swath_chan_file[1] = image_m_nav_crefl_04.real4.3200.768 swath_chan_file[2] = image_m_nav_crefl_03.real4.3200.768 grid_cols = 15262 grid_rows = 4408 grid_chan_file[0] = result_m_nav_crefl_01_eqc_fit.real4.15262.4408 grid_chan_file[1] = result_m_nav_crefl_04_eqc_fit.real4.15262.4408 grid_chan_file[2] = result_m_nav_crefl_03_eqc_fit.real4.15262.4408 maximum_weight_mode = 0 chan_scan_first = 0 colrow_scan_first = 0 grid_col_start = 0 grid_row_start = 0 swath_data_type[0] = f4 swath_data_type[1] = f4 swath_data_type[2] = f4 grid_data_type[0] = f4 grid_data_type[1] = f4 grid_data_type[2] = f4 swath_fill[0] = -999.000000 swath_fill[1] = -999.000000 swath_fill[2] = -999.000000 grid_fill[0] = -999.000000 grid_fill[1] = -999.000000 grid_fill[2] = -999.000000
col_row_min = 0.000000e+00 weight_count = 10000 weight_min = 0.010000 weight_distance_max = 2.000000 weight_delta_max = 40.000000 weight_sum_min = 0.010000 swath_col_offset = 0.000000 swath_row_offset = 0.000000 fill count[0]: 47801742 fill count[1]: 47801742 fill count[2]: 47801742 DEBUG : Fornav successfully completed for grid eqc_fit, corrected_reflectance data DEBUG : Inheriting grid eqc_fit's information from lower-resolution navigation INFO : Running ll2cr for grid eqc_fit and bands [('crefl', '08')] DEBUG : Real upper-left corner (-1.302259,38.436245) DEBUG : Real lower-right corner (-35.577479,28.536811) INFO : row and column calculation complete DEBUG : Checking stem 'result_i_nav_crefl_08_eqc_fit' DEBUG : Running fornav with 'fornav 1 -v -t f4 -f -999.0 -F -999.0 -D 40 -d 2 -s 0 0 6400 48 32 ll2cr_i_nav_eqc_fit_cols.real4.6400.1536 ll2cr_i_nav_eqc_fit_rows.real4.6400.1536 image_i_nav_crefl_08.real4.6400.1536 15262.0 4408.0 result_i_nav_crefl_08_eqc_fit.real4.15262.4408' fornav: $Header: /disks/megadune/data/tharan/ms2gth/src/fornav/fornav.c,v 1.32 2012/07/24 21:42:17 tharan Exp $ chan_count = 1 swath_cols = 6400 swath_scans = 48 swath_rows_per_scan = 32 swath_col_file = ll2cr_i_nav_eqc_fit_cols.real4.6400.1536 swath_row_file = ll2cr_i_nav_eqc_fit_rows.real4.6400.1536 swath_chan_file[0] = image_i_nav_crefl_08.real4.6400.1536 grid_cols = 15262 grid_rows = 4408 grid_chan_file[0] = result_i_nav_crefl_08_eqc_fit.real4.15262.4408
maximum_weight_mode = 0 chan_scan_first = 0 colrow_scan_first = 0 grid_col_start = 0 grid_row_start = 0 swath_data_type[0] = f4 grid_data_type[0] = f4 swath_fill[0] = -999.000000 grid_fill[0] = -999.000000
col_row_min = 0.000000e+00 weight_count = 10000 weight_min = 0.010000 weight_distance_max = 2.000000 weight_delta_max = 40.000000 weight_sum_min = 0.010000 swath_col_offset = 0.000000 swath_row_offset = 0.000000
fill count[0]: 67274896 On next call to fornav, use: chan_scan_first: -1 colrow_scan_first: -1 swath_scans: 0 Removing file: result_i_nav_crefl_08_eqc_fit.real4.15262.4408 ERROR : Couldn't find fornav output file 'result_i_nav_crefl_08_eqc_fit.real4.15262.4408' WARNING : fornav failed for i_nav band, grid eqc_fit, remapping as corrected_reflectance DEBUG : Exception was: Traceback (most recent call last): File "/packages/cspp/polar2grid/ShellB3/lib/python2.7/site-packages/polar2grid-1.1.6-py2.7.egg/polar2grid/remap.py", line 305, in run_fornav fornav_dict = fornav_job["result"].get() File "/packages/cspp/polar2grid/ShellB3/lib/python2.7/multiprocessing/pool.py", line 528, in get raise self._value RuntimeError: Couldn't find fornav output file 'result_i_nav_crefl_08_eqc_fit.real4.15262.4408'