search

geodesymiami / rsmas_insar

RSMAS InSAR code
https://rsmas-insar.readthedocs.io/
GNU General Public License v3.0
59 stars 23 forks source link

Why stackSentinel does not exclude dates with missing frames? #490

Open falkamelung opened 3 years ago

falkamelung commented 3 years ago

The date 20190917 in KokoxiliChunk41SenDT19 is flagged by stackSentinel as dates for which it looks there are missing scenes but it is still included in the runfiles. How to handle this? Is this a bug in stackSentinel? This could be dealt with by checking for FileNotFoundError: [Errno 2] No such file or directory: '/tmp/secondarys/20190917/IW2.xml' in the run_04*.e file

cat run_04_fullBurst_geo2rdr_7_20190917_1.o
API open (WR): /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/range_01.off
API open (WR): /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/azimuth_01.off
GDAL open (R): /tmp/geom_reference/IW1/hgt_01.rdr.vrt
GDAL open (R): /tmp/geom_reference/IW1/lat_01.rdr.vrt
GDAL open (R): /tmp/geom_reference/IW1/lon_01.rdr.vrt
API close:  /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/range_01.off
API open (R): /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/range_01.off
API close:  /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/range_01.off
API close:  /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/azimuth_01.off
API open (R): /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/azimuth_01.off
API close:  /scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917/IW1/azimuth_01.off
GDAL close: /tmp/geom_reference/IW1/lat_01.rdr.vrt
GDAL close: /tmp/geom_reference/IW1/lon_01.rdr.vrt
GDAL close: /tmp/geom_reference/IW1/hgt_01.rdr.vrt
Completed Parsing the Configuration file
Functions to be executed:
['Function-1']
Running: geo2rdr
['--secondary', '/tmp/secondarys/20190917', '--reference', '/tmp/reference', '--geom_referenceDir', '/tmp/geom_reference', '--coregSLCdir', '/scratch1/05861/tg851601/KokoxiliChunk41SenDT19/coreg_secondarys/20190917', '--azimuth_misreg', '/scratch1/05861/tg851601/KokoxiliChunk41SenDT19/misreg/azimuth/dates/20190917.txt', '--range_misreg', '/scratch1/05861/tg851601/KokoxiliChunk41SenDT19/misreg/range/dates/20190917.txt']
This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
CPU mode
Estimated burst offset:  0
Overlap pair 0: Burst 0 of reference matched with Burst 0 of secondary
Burst 0 of reference matched with Burst 0 of secondary
Additional time offset applied in geo2rdr: 0.0 secs
Additional range offset applied in geo2rdr: 0.0 m
Turning off bistatic delay correction term by default.
 Orbit interpolation method: hermite
 threads           4
 Starting Acquisition time:    900.51349600000003     
 Stop Acquisition time:    903.59683045000008     
 Azimuth line spacing in secs:    2.0555562999999980E-003
 Near Range in m:    801096.60321629210     
 Far  Range in m:    851072.69926327991     
 Range sample spacing in m:    2.3295621147153232     
 Radar Image Lines:         1501
 Radar Image Width:        21454
 reading dem ...
 Geocoded Lines:          1499
 Geocoded Samples:       21577
 Dopplers:    0.0000000000000000        0.0000000000000000     
 geo2rdr on            4  threads...
 Processing line:            1           0
 Processing line:         1001      573000
 Number of pixels outside the image:       858927
 Number of pixels with valid data:       30485650
 Number of pixels that converged:           20394
 elapsed time =    27.4570312      seconds
cat run_04_fullBurst_geo2rdr_7_20190917_1.e
Traceback (most recent call last):
  File "/tmp/rsmas_insar/3rdparty/miniconda3/share/isce2/topsStack/SentinelWrapper.py", line 251, in <module>
    main(args.start,args.end)
  File "/tmp/rsmas_insar/3rdparty/miniconda3/share/isce2/topsStack/SentinelWrapper.py", line 242, in main
    cfgParser.runCmd()
  File "/tmp/rsmas_insar/3rdparty/miniconda3/share/isce2/topsStack/SentinelWrapper.py", line 54, in runCmd
    func_modules.main(self.funcParams[section])
  File "/tmp/rsmas_insar/3rdparty/miniconda3/share/isce2/topsStack/geo2rdr.py", line 226, in main
    secondary = ut.loadProduct(os.path.join(inps.secondary, 'IW{0}.xml'.format(swath)))
  File "/tmp/rsmas_insar/3rdparty/miniconda3/share/isce2/topsStack/s1a_isce_utils.py", line 26, in loadProduct
    obj = pm.loadProduct(xmlname)
  File "/tmp/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/isce/components/iscesys/Component/ProductManager.py", line 25, in loadProduct
    self.load(filename)
  File "/tmp/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/isce/components/iscesys/Component/Configurable.py", line 1407, in load
    tmpProp, tmpFact, tmpMisc = FP.parse(filename)
  File "/tmp/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
    root = ET.parse(filename)
  File "/tmp/rsmas_insar/3rdparty/miniconda3/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/tmp/rsmas_insar/3rdparty/miniconda3/lib/python3.8/xml/etree/ElementTree.py", line 584, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/secondarys/20190917/IW2.xml'
*****************************************
Number of dates : 155
List of dates :
['20141007', '20141031', '20141124', '20141218', '20150111', '20150204', '20150312', '20150405', '20150429', '20150523', '20150616', '20150827', '20151107', '20160306', '20160330', '20160423', '20160517', '20160610', '20160704', '20160728', '20160821', '20160914', '20161014', '20161107', '20161201', '20161225', '20170118', '20170211', '20170301', '20170313', '20170325', '20170412', '20170424', '20170506', '20170518', '20170530', '20170611', '20170623', '20170705', '20170717', '20170729', '20170810', '20170822', '20170915', '20170927', '20171009', '20171021', '20171102', '20171114', '20171126', '20171208', '20171220', '20180101', '20180113', '20180125', '20180206', '20180218', '20180302', '20180314', '20180326', '20180407', '20180419', '20180501', '20180513', '20180525', '20180606', '20180618', '20180712', '20180724', '20180817', '20180829', '20180910', '20180922', '20181004', '20181016', '20181028', '20181203', '20181215', '20181227', '20190108', '20190120', '20190201', '20190213', '20190225', '20190309', '20190321', '20190402', '20190414', '20190426', '20190508', '20190520', '20190601', '20190613', '20190625', '20190707', '20190719', '20190731', '20190812', '20190824', '20190905', '20190917', '20190929', '20191011', '20191023', '20191104', '20191116', '20191128', '20191210', '20191222', '20200103', '20200115', '20200127', '20200208', '20200220', '20200303', '20200315', '20200327', '20200408', '20200420', '20200502', '20200514', '20200526', '20200607', '20200619', '20200701', '20200713', '20200725', '20200806', '20200818', '20200830', '20200911', '20200923', '20201005', '20201017', '20201029', '20201110', '20201122', '20201204', '20201216', '20201228', '20210109', '20210121', '20210202', '20210214', '20210226', '20210310', '20210322', '20210403', '20210415', '20210427', '20210509', '20210521', '20210602', '20210614', '20210626']
date      south      north
20141007 39.410606 42.916508
20141031 39.411541 42.917347
20141124 39.411861 42.91737
...
...
20190917 39.133347 44.021729
...
...

dates for which it looks like there are missing frames
['20190917']

The reference date was not chosen. The first date is considered as reference date.

All SLCs will be coregistered to : 20141007
secondary dates :
['20141031', '20141124', '20141218', '20150111', '20150204', '20150312', '20150405', '20150429', '20150523', '20150616', '20150827', '20151107', '20160306', '20160330', '20160423', '20160517', '20160610', '20160728', '20160821', '20160914', '20161014', '20161107', '20161201', '20161225', '20170118', '20170211', '20170301', '20170313', '20170325', '20170412', '20170424', '20170506', '20170518', '20170530', '20170611', '20170623', '20170705', '20170717', '20170729', '20170810', '20170822', '20170915', '20170927', '20171009', '20171021', '20171102', '20171114', '20171126', '20171208', '20171220', '20180101', '20180113', '20180125', '20180206', '20180218', '20180302', '20180314', '20180326', '20180407', '20180419', '20180501', '20180513', '20180525', '20180606', '20180618', '20180712', '20180724', '20180817', '20180829', '20180910', '20180922', '20181004', '20181016', '20181028', '20181203', '20181215', '20181227', '20190108', '20190120', '20190201', '20190213', '20190225', '20190309', '20190321', '20190402', '20190414', '20190426', '20190508', '20190520', '20190601', '20190613', '20190625', '20190707', '20190719', '20190731', '20190812', '20190824', '20190905', '20190917', '20190929', '20191011', '20191023', '20191104', '20191116', '20191128', '20191210', '20191222', '20200103', '20200115', '20200127', '20200208', '20200220', '20200303', '20200315', '20200327', '20200408', '20200420', '20200502', '20200514', '20200526', '20200607', '20200619', '20200701', '20200713', '20200725', '20200806', '20200818', '20200830', '20200911', '20200923', '20201005', '20201017', '20201029', '20201110', '20201122', '20201204', '20201216', '20201228', '20210109', '20210121', '20210202', '20210214', '20210226', '20210310', '20210322', '20210403', '20210415', '20210427', '20210509', '20210521', '20210602', '20210614', '20210626']

A subsection works fine, as `stackSentinel` realizes that 
run02errunknown1_KokoxiliChunk41SenDT19

Number of dates : 4 List of dates : ['20190824', '20190905', '20190917', '20190929'] date south north 20190824 40.51849 44.022385 20190905 40.518333 44.021854 20190917 42.015282 44.021729 20190929 40.518448 44.022243

The overlap region among all dates (based on the preview kml files): South North East West 42.015282 44.021729 86.154984 89.590729

All dates (4) ['20190824', '20190905', '20190917', '20190929']

dates covering the bbox (3) ['20190824', '20190905', '20190929']