search

isce-framework / isce2

InSAR Scientific Computing Environment version 2
Other
480 stars 244 forks source link

Processing ICEYE sample SLC data fails #515

Open EJFielding opened 2 years ago

EJFielding commented 2 years ago

I am trying to process one of the ICEYE sample datasets, from their "Daily Coherent Ground Track Repeat Stack" dataset. I found there is an ICEYE_SLC sensor module that looks like should read their HDF5 files. When I use that sensor with stripmapApp.py, it has trouble with the look direction. I don't know whether they changed their file format since this module was written three years ago. This is the output:

This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
2022-07-03 21:07:08,586 - isce.insar - INFO - ISCE VERSION = 2.5.3, RELEASE_SVN_REVISION = ,RELEASE_DATE = 20210823, CURRENT_SVN_REVISION =
ISCE VERSION = 2.5.3, RELEASE_SVN_REVISION = ,RELEASE_DATE = 20210823, CURRENT_SVN_REVISION =
2022-07-03 21:07:08,588 - root - INFO - reference sensor not provided explicitly, using common sensor
2022-07-03 21:07:08,606 - h5py._conv - DEBUG - Creating converter from 7 to 5
2022-07-03 21:07:08,607 - h5py._conv - DEBUG - Creating converter from 5 to 7
2022-07-03 21:07:08,607 - h5py._conv - DEBUG - Creating converter from 7 to 5
2022-07-03 21:07:08,607 - h5py._conv - DEBUG - Creating converter from 5 to 7
2022-07-03 21:07:08,655 - root - INFO - secondary sensor not provided explicitly, using common sensor
do_unwrap  True
Step processing
Running step startup

    Insar Application:
    Implements InSAR processing flow for a pair of scenes from
    sensor raw data to geocoded, flattened interferograms.

The currently supported sensors are:  ['ALOS', 'ALOS2', 'ALOS_SLC', 'COSMO_SKYMED', 'COSMO_SKYMED_SLC', 'ENVISAT', 'ENVISAT_SLC', 'ERS', 'ERS_ENVISAT', 'ERS_ENVISAT_SLC', 'ERS_SLC', 'ICEYE_SLC', 'KOMPSAT5', 'RADARSAT1', 'RADARSAT2', 'RISAT1', 'RISAT1_SLC', 'ROI_PAC', 'SAOCOM_SLC', 'SENTINEL1', 'SICD_RGZERO', 'TERRASARX', 'UAVSAR_HDF5_SLC', 'UAVSAR_POLSAR', 'UAVSAR_RPI', 'UAVSAR_STACK']
Dumping the application's pickle object _insar to file  PICKLE/startup
The remaining steps are (in order):  ['preprocess', 'cropraw', 'formslc', 'cropslc', 'verifyDEM', 'topo', 'geo2rdr', 'coarse_resample', 'misregistration', 'refined_resample', 'dense_offsets', 'rubber_sheet_range', 'rubber_sheet_azimuth', 'fine_resample', 'split_range_spectrum', 'sub_band_resample', 'interferogram', 'sub_band_interferogram', 'filter', 'filter_low_band', 'filter_high_band', 'unwrap', 'unwrap_low_band', 'unwrap_high_band', 'ionosphere', 'geocode', 'geocodeoffsets', 'endup']
Running step preprocess
DOPPLER:  ICEYE_SLC useDEFAULT
Reference data is in SLC format. Adding _slc to output name.
2022-07-03 21:07:08,901 - isce.make_raw - INFO - Extracting image
2022-07-03 21:07:08,902 - h5py._conv - DEBUG - Creating converter from 3 to 5
Traceback (most recent call last):
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/applications/stripmapApp.py", line 1002, in <module>
    status = insar.run()
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/iscesys/Component/Application.py", line 142, in run
    exitStatus = self._processSteps()
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/iscesys/Component/Application.py", line 405, in _processSteps
    result = func(*pargs, **kwargs)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/StripmapProc/Factories.py", line 46, in __call__
    return self.method(self.other, *args, **kwargs)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/StripmapProc/runPreprocessor.py", line 89, in runPreprocessor
    reference = make_raw(sensor, referencedop)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/StripmapProc/runPreprocessor.py", line 206, in make_raw
    objMakeRaw(sensor=sensor, doppler=doppler)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/iscesys/Component/Component.py", line 179, in __call__
    return getattr(self, self.__class__.__name__.lower())(*args)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/applications/make_raw.py", line 235, in make_raw
    self.sensor.extractImage()
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/Sensor/ICEYE_SLC.py", line 229, in extractImage
    self.parse()
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/Sensor/ICEYE_SLC.py", line 121, in parse
    self.populateMetadata(fp)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/Sensor/ICEYE_SLC.py", line 129, in populateMetadata
    self._populatePlatform(file)
  File "/u/pez0/fielding/tools/ISCE2_test/install/isce/components/isceobj/Sensor/ICEYE_SLC.py", line 140, in _populatePlatform
    platform.setPointingDirection(self.lookMap[file['look_side'][()].upper()])
KeyError: b'RIGHT'
xiaoming-sar commented 1 year ago

For those who may still be interested, the change in format of the attribute ['look_side'] in ICEYE SLC .h5 file, from string to byte, might be the cause of the issue. To fix it, simply modify line 140 of install/isce/components/isceobj/Sensor/ICEYE_SLC.py from

platform.setPointingDirection(self.lookMap[file['look_side'][()].upper()]) to platform.setPointingDirection(self.lookMap[file['look_side'][()].upper().decode()])

EJFielding commented 1 year ago

Thanks for the suggestion!

kamilapawluszek commented 11 months ago

Dear @EJFielding have you fixed the problem of ICEYE processing in ISCE? Have you succeeded in SAR interferometry with this data? Cheers

EJFielding commented 11 months ago

No, I have not had time to go back to looking at ICEYE data.

StevenBickerton commented 6 months ago

This thread is a bit stale now, but I've actually been trying to run the same ICEYE GTR sample through ISCE2 - 2.6.3. I encountered the same issue described above, and the suggested fix did indeed work for me. However, I'm not able to get stripmapApp.py to run to completion. It fails on the geocode step, with the following error:

GDAL open (R): interferogram/phsig.cor.vrt
ERROR 1: Invalid dataset dimensions : 236 x -1

I'd be interested in any relevant information on this. I'm happy to post it as a new issue, or leave it here as an extension of 'getting iceye gtr sample working'.